flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_workspace_bytes¶
- flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_workspace_bytes(T: int, device: device, *, hidden: int = 4096) int¶
Bytes of the caller-owned backward workspace for up to
Ttokens ondevice.This package’s backward layout: a 256-byte ticket region and 512 bytes of column-slab arrival counts first (zero on first use; every completed launch leaves them zero), the deterministic package’s regions without its per-chunk FP32
dfnpartials, then the column slab’s per-chunk FP32dfnslots (min(32, ceil(T / 64)) x 24 x 4H x 4bytes at hidden sizeH). A deterministic backward workspace (cake_mhc_det_train_backward_workspace()) does not serve this backward.- Parameters:
T (int) – Largest token count the buffer serves (the size is the maximum over every token count up to
T);T == 0needs no workspace.device (torch.device) – CUDA device of the calls.
hidden (int) – Hidden size of the calls the buffer serves (default
4096), as incake_mhc_nondet_train_forward_workspace_bytes().
- Returns:
The size of the backward workspace in bytes.
- Return type:
int