flashinfer.cake_kda_nondet_train.cake_kda_nondet¶
- flashinfer.cake_kda_nondet_train.cake_kda_nondet(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, *, scale: float | None = None, workspace: Any | None = None, save_mode: str | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False, checkpoint_handoff: str = 'auto') Tensor¶
Packed KDA chunked training as an autograd node (
torch.utils.checkpointcompatible).Returns
outBF16 likevwith gradients toq, k, v, g, beta; seecake_kda_nondet_train_forward()for the operands,save_mode,arena,doc_lengths(the node records the option and the document lengths, so a backward that allocates its own workspace takes them too) and the raw gate mode (use_gate_in_kernel=True,A_log,dt_bias,lower_bound: gradients to the raw gateg, BF16, and toA_log/dt_bias, FP32), the in-kernel Q/K L2 norm (use_qk_l2norm_in_kernel=True: rawq/kin, their gradients out) andcake_kda_nondet_train_backward()for the gradients. The node’s backward is this package’s.checkpoint_handoff("auto","on"or"off"; another value raisesValueError) is the checkpoint hand-off undertorch.utils.checkpoint(the module docstring; the raw gate mode and the in-kernel Q/K L2 norm never engage it).- Parameters:
q (torch.Tensor) – BF16 packed rows
[1, S, H, 128](or[S, H, 128]), as incake_kda_nondet_train_forward().k (torch.Tensor) – BF16 packed rows
[1, S, H, 128](or[S, H, 128]), as incake_kda_nondet_train_forward().v (torch.Tensor) – BF16 packed rows
[1, S, H, 128](or[S, H, 128]), as incake_kda_nondet_train_forward().g (torch.Tensor) – FP32 activated decay gate; raw BF16 gate when
use_gate_in_kernel=True.beta (torch.Tensor) – BF16
[1, S, H]delta-rule gate after the sigmoid.cu_seqlens (torch.Tensor) – int32 device document offsets, first 0 and last
S.scale (float, optional) – Output scale;
128 ** -0.5whenNone.workspace (optional) – Caller-owned workspace from
cake_kda_nondet_train_workspace().save_mode (str, optional) –
"light"or"tape";Noneuses the workspace’s default ("light"without one).arena (optional) – Lower-memory arena option from
cake_kda_nondet_train_arena().doc_lengths (sequence of int, optional) – Host document lengths in offset order; required by a batched arena.
use_gate_in_kernel (bool, default=False) – Activate the raw BF16 gate inside the kernels.
A_log (torch.Tensor, optional) – Contiguous FP32
[H]raw-gate parameter.dt_bias (torch.Tensor, optional) – Contiguous FP32
[H * 128]or[H, 128]raw-gate bias.lower_bound (float, optional) – Required in raw-gate mode, with
-5 <= lower_bound < 0.use_qk_l2norm_in_kernel (bool, default=False) – Normalize raw query and key rows inside the kernels; input layout rules are those of
cake_kda_nondet_train_forward().checkpoint_handoff (str, default="auto") – Checkpoint hand-off mode:
"auto","on"or"off".