flashinfer.cake_kda_nondet_train.cake_kda_nondet

flashinfer.cake_kda_nondet_train.cake_kda_nondet(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, *, scale: float | None = None, workspace: Any | None = None, save_mode: str | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False, checkpoint_handoff: str = 'auto') → Tensor

Packed KDA chunked training as an autograd node (torch.utils.checkpoint compatible).

Returns out BF16 like v with gradients to q, k, v, g, beta; see cake_kda_nondet_train_forward() for the operands, save_mode, arena, doc_lengths (the node records the option and the document lengths, so a backward that allocates its own workspace takes them too) and the raw gate mode (use_gate_in_kernel=True, A_log, dt_bias, lower_bound: gradients to the raw gate g, BF16, and to A_log / dt_bias, FP32), the in-kernel Q/K L2 norm (use_qk_l2norm_in_kernel=True: raw q / k in, their gradients out) and cake_kda_nondet_train_backward() for the gradients. The node’s backward is this package’s. checkpoint_handoff ("auto", "on" or "off"; another value raises ValueError) is the checkpoint hand-off under torch.utils.checkpoint (the module docstring; the raw gate mode and the in-kernel Q/K L2 norm never engage it).

Parameters:
  • q (torch.Tensor) – BF16 packed rows [1, S, H, 128] (or [S, H, 128]), as in cake_kda_nondet_train_forward().

  • k (torch.Tensor) – BF16 packed rows [1, S, H, 128] (or [S, H, 128]), as in cake_kda_nondet_train_forward().

  • v (torch.Tensor) – BF16 packed rows [1, S, H, 128] (or [S, H, 128]), as in cake_kda_nondet_train_forward().

  • g (torch.Tensor) – FP32 activated decay gate; raw BF16 gate when use_gate_in_kernel=True.

  • beta (torch.Tensor) – BF16 [1, S, H] delta-rule gate after the sigmoid.

  • cu_seqlens (torch.Tensor) – int32 device document offsets, first 0 and last S.

  • scale (float, optional) – Output scale; 128 ** -0.5 when None.

  • workspace (optional) – Caller-owned workspace from cake_kda_nondet_train_workspace().

  • save_mode (str, optional) – "light" or "tape"; None uses the workspace’s default ("light" without one).

  • arena (optional) – Lower-memory arena option from cake_kda_nondet_train_arena().

  • doc_lengths (sequence of int, optional) – Host document lengths in offset order; required by a batched arena.

  • use_gate_in_kernel (bool, default=False) – Activate the raw BF16 gate inside the kernels.

  • A_log (torch.Tensor, optional) – Contiguous FP32 [H] raw-gate parameter.

  • dt_bias (torch.Tensor, optional) – Contiguous FP32 [H * 128] or [H, 128] raw-gate bias.

  • lower_bound (float, optional) – Required in raw-gate mode, with -5 <= lower_bound < 0.

  • use_qk_l2norm_in_kernel (bool, default=False) – Normalize raw query and key rows inside the kernels; input layout rules are those of cake_kda_nondet_train_forward().

  • checkpoint_handoff (str, default="auto") – Checkpoint hand-off mode: "auto", "on" or "off".