flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace¶
- flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace(device: device | str | int | None = None, *, budget_bytes: int | None = None) Any¶
A workspace (
CakeGQANondetWorkspace) forcake_gqa_nondet()and its two halves.It holds the forward’s tile counter and the backward scratch, grown to the largest shape seen and never shrunk (
.nbytes;.release()drops the scratch). The backward’s dS^T workspace and row-map window gradients are allocated per call from PyTorch’s caching allocator and freed when the call returns. One workspace serves one stream at a time.- Parameters:
device (torch.device, str or int, optional) – The CUDA device;
Noneis the current device.budget_bytes (int, optional) – The bound of one backward call’s dS^T workspace;
Noneis 16 GiB. A call whose dS^T exceeds it runs in chunks (at least one KV head and two key blocks per chunk, even past the budget). The chunks set the dQ rounding: one budget gives the same bits on every run.
- Returns:
A
CakeGQANondetWorkspace.- Return type:
CakeGQANondetWorkspace