flashinfer.cake_mhc_det_train.cake_mhc_det_train_backward_workspace_bytes¶
- flashinfer.cake_mhc_det_train.cake_mhc_det_train_backward_workspace_bytes(T: int, device: device, *, hidden: int = 4096) int¶
Bytes of the caller-owned backward workspace for up to
Ttokens of hidden sizehidden.The
backward_finishroute holds one 256-byte ticket region (first, at the same offset for every token count), the transposed BF16 pieces offn, the per-token records of the token-owner pass (dz'pieces and the per-token scalars, padded to a multiple of 128 rows), the per-CTA parameter partials and the per-chunk FP32 partials ofdfn. The ticket words must be zero on first use (allocate withtorch.zerosorcake_mhc_det_train_backward_workspace()); the kernels leave them zero.- Parameters:
T (int) – Largest token count the buffer serves (the size is the maximum over every token count up to
T);T == 0needs no workspace.device (torch.device) – CUDA device of the calls.
hidden (int) – Hidden size
Hof the calls the buffer serves (X.shape[2]; default4096), as incake_mhc_det_train_forward_workspace_bytes(): a buffer sized for oneHis refused by a call of a largerHit is too small for.
- Returns:
The size of the backward workspace in bytes.
- Return type:
int