flashinfer.cake_kda_nondet_train

Cake non-deterministic KDA training entry points. The module is a stable API; its backend package, flashinfer.experimental.cake.cake_kda_nondet_train, is experimental and is imported on first use.

cake_kda_nondet_train_forward(q, k, v, g, ...)

Packed KDA chunked training forward.

cake_kda_nondet_train_backward(q, k, v, g, ...)

Packed KDA chunked training backward.

cake_kda_nondet(q, k, v, g, beta, cu_seqlens, *)

Packed KDA chunked training as an autograd node (torch.utils.checkpoint compatible).

cake_kda_nondet_train_arena(*[, lean, ...])

The lower-memory arena option (a CakeKDANondetArena; the default, lean=False, tail_passes=1, no batching, is the full layout).

cake_kda_nondet_train_workspace(S, num_docs, ...)

A caller-owned CakeKDANondetWorkspace (one arena of internal intermediates) for S tokens in num_docs documents with H heads on device, reused by every call that receives it: the forward layout only with backward=False; save_mode is the default of the calls on it and sizes the arena for that mode; arena (cake_kda_nondet_train_arena()) is the lower-memory arena option, off when None; under a batched option doc_lengths (the documents' token counts) sizes the arena for the exact document groups instead of the bound of every group; use_gate_in_kernel / use_qk_l2norm_in_kernel are the gate and Q/K L2 norm modes of the calls it serves (a call in another mode is refused).

cake_kda_nondet_train_workspace_bytes(S, ...)

Bytes cake_kda_nondet_train_workspace() allocates for the problem (sm_count: the device's SM count, the current CUDA device's when None; it sets the segment count of the segmented recurrences; arena: the lower-memory arena option, off when None; doc_lengths: the exact document groups of a batched option, else their bound; use_gate_in_kernel: the gate mode, whose backward layouts carry no gate epilogue buffers in raw mode; use_qk_l2norm_in_kernel: the Q/K L2 norm mode, likewise without those buffers).

cake_kda_nondet_train_saved_bytes(S, H[, ...])

Bytes of the saved set: Aqk and Akk (256 * S * H), plus, when num_docs is given, the device tables and the version's per-call tensors: the forward's gated queries qg (and, on the rows whose backward reads it from the set, the WY representation w / u) and the segment-boundary states under the size rule with save_mode="light"; the state tape, v_new and the segment transfers with save_mode="tape"; with use_qk_l2norm_in_kernel=True the FP32 reciprocal norms [2, S, H] (8 * S * H) in either.

is_cake_kda_nondet_train_supported([device])

Whether device (the current CUDA device by default) is a target of the Cake kernels (compute capability 10.0, 10.3 or 10.7).