flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace

flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace(device: device | str | int | None = None, *, budget_bytes: int | None = None) → Any

A workspace (CakeGQANondetWorkspace) for cake_gqa_nondet() and its two halves.

It holds the forward’s tile counter and the backward scratch, grown to the largest shape seen and never shrunk (.nbytes; .release() drops the scratch). The backward’s dS^T workspace and row-map window gradients are allocated per call from PyTorch’s caching allocator and freed when the call returns. One workspace serves one stream at a time.

Parameters:
  • device (torch.device, str or int, optional) – The CUDA device; None is the current device.

  • budget_bytes (int, optional) – The bound of one backward call’s dS^T workspace; None is 16 GiB. A call whose dS^T exceeds it runs in chunks (at least one KV head and two key blocks per chunk, even past the budget). The chunks set the dQ rounding: one budget gives the same bits on every run.

Returns:

A CakeGQANondetWorkspace.

Return type:

CakeGQANondetWorkspace