flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_workspace_bytes

flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_workspace_bytes(T: int, device: device, *, hidden: int = 4096) → int

Bytes of the caller-owned forward workspace for up to T tokens on device.

The forward workspace has one layout in both packages (see cake_mhc_det_train_forward_workspace_bytes()).

Parameters:
  • T (int) – Largest token count the buffer serves (the size is the maximum over every token count up to T); T == 0 needs no workspace.

  • device (torch.device) – CUDA device of the calls.

  • hidden (int) – Hidden size H of the calls the buffer serves (X.shape[2]; default 4096): 4096, 5120 or 7168 on SM100 / SM103, 4096 on SM107 (any other value raises ValueError).

Returns:

The size of the forward workspace in bytes.

Return type:

int