flashinfer.cake_mhc_det_train.cake_mhc_det_train_forward_workspace_bytes

flashinfer.cake_mhc_det_train.cake_mhc_det_train_forward_workspace_bytes(T: int, device: device, *, hidden: int = 4096) → int

Bytes of the caller-owned forward workspace for up to T tokens of hidden size hidden.

The forward routes (forward_prepare and the forward_stats replay) hold one 256-byte ticket region (first, at the same offset for every token count), the BF16 pieces of fn in the mapping operand layout and the FP32 per-K-unit partials of the mapping pass. The ticket words must be zero on first use (allocate with torch.zeros or cake_mhc_det_train_forward_workspace()); the kernels leave them zero.

Parameters:
  • T (int) – Largest token count the buffer serves. The size is the maximum over every token count up to T, so the buffer also serves every smaller one (and CUDA graph replays at a smaller T); T == 0 needs no workspace (nothing is launched).

  • device (torch.device) – CUDA device of the calls.

  • hidden (int) – Hidden size H of the calls the buffer serves (X.shape[2]; default 4096): a hidden size the device’s architecture exports (4096, 5120 or 7168 on SM100 / SM103 / SM107), else ValueError. The fn pieces and the partials scale with 4 H; every call checks the buffer against its own H and T, so a buffer sized for one H is refused (ValueError) by a call of a larger H it is too small for.

Returns:

The size of the forward workspace in bytes.

Return type:

int