flashinfer.cake_gqa_nondet_train

Cake non-deterministic GQA attention training for head dimension 256: a varlen causal forward and backward for SM100a, SM103a and SM107a. The module is a stable API; its backend package, flashinfer.experimental.cake.cake_gqa_nondet_train, is experimental and is imported on first use.

The mode is part of every name, and no entry takes a deterministic keyword. Every sum of the forward and the backward has one fixed order: the backward stores dS^T into a workspace chunk sized by the workspace’s budget_bytes (default 16 GiB), sums dK and dV in tensor memory, computes dQ as a GEMM over the workspace and folds row-map destinations in window order. out, lse, dQ and every dK / dV form therefore repeat bit for bit run to run for one shape, one max_seqlen_k and one budget, and nothing raises PyTorch’s determinism alert.

cake_gqa_nondet is the autograd entry (its Function is CakeGQANondetFunction); the forward and backward halves, the workspace and its size are separate entries.

cake_gqa_nondet(q, k, v, cu_seqlens_q, ...)

Packed-varlen causal GQA attention, head dimension 256, with autograd (CakeGQANondetFunction).

cake_gqa_nondet_train_forward(q, k, v, ...)

Forward without autograd, causal with bottom-right alignment per segment.

cake_gqa_nondet_train_backward(q, k, v, out, ...)

Backward without autograd: (dq, dk, dv) from the forward's out and lse.

cake_gqa_nondet_train_workspace([device, ...])

A workspace (CakeGQANondetWorkspace) for cake_gqa_nondet() and its two halves.

cake_gqa_nondet_train_workspace_bytes(...[, ...])

Bytes of device memory one backward call needs beside its operands and outputs.