flashinfer.cake_kda_det_train.cake_kda_det_train_forward

flashinfer.cake_kda_det_train.cake_kda_det_train_forward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, *, scale: float | None = None, save_for_backward: bool = True, workspace: Any | None = None, save_mode: str | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False) → tuple[Tensor, Any | None]

Packed KDA chunked training forward.

Parameters:
  • q (torch.Tensor) – BF16 [1, S, H, 128] (or [S, H, 128]), contiguous.

  • k (torch.Tensor) – BF16 [1, S, H, 128] (or [S, H, 128]), contiguous.

  • v (torch.Tensor) – BF16 [1, S, H, 128] (or [S, H, 128]), contiguous.

  • g (torch.Tensor) – FP32 [1, S, H, 128] natural-log decay gate (preactivated); with use_gate_in_kernel=True the raw gate, BF16 [1, S, H, 128], which may be a strided view (unit innermost stride, token and head strides multiples of 8 elements, rows that do not overlap, a 16-byte aligned address; never copied).

  • beta (torch.Tensor) – BF16 [1, S, H] delta-rule gate after the sigmoid.

  • cu_seqlens (torch.Tensor) – int32 [num_docs + 1] document offsets on the device (first 0, last S).

  • scale (float, optional) – Output scale; 128 ** -0.5 when None.

  • save_for_backward (bool) – Write the saved set (Aqk / Akk on the token rows plus the device chunk tables).

  • workspace (CakeKDADetWorkspace, optional) – A reused workspace from cake_kda_det_train_workspace(); without one this call allocates its own.

  • save_mode (str, optional) – "light" (saved set version 4: Aqk / Akk and the tables, 256 B per token-head, the forward’s gated queries qg, 256 B per internal token-head, plus the segment-boundary states on the segmented route when S >= 384 * NSEG, so the backward skips the forward half of its recurrence recompute, and the WY representation on the rows whose backward reads it from the set) or "tape" (version 2: also the forward’s state tape and v_new, 768 B more per token-head, so the backward skips the whole recompute); None takes the workspace’s default ("light" without one).

  • arena (CakeKDADetArena, optional) – The lower-memory arena option (cake_kda_det_train_arena(); off by default): the forward’s layout is the same under every option, and a batched option (batch_tokens) runs the forward once per document group; None takes the workspace’s option, a differing option on a given workspace is refused.

  • doc_lengths (sequence of int, optional) – The documents’ token counts in cu_seqlens order (host integers; len == num_docs, sum == S), required under a batched option (the host reads no tensor) and checked against cu_seqlens on the device.

  • use_gate_in_kernel (bool) – False (the default): g is the activated FP32 gate and A_log, dt_bias, lower_bound must be None. True: g is the raw BF16 gate and the kernels activate it as lower_bound * sigmoid(exp(A_log) * (g + dt_bias)); the workspace must be allocated with use_gate_in_kernel=True and the saved set records the mode.

  • A_log (torch.Tensor, optional) – FP32 [H], contiguous (raw gate mode).

  • dt_bias (torch.Tensor, optional) – FP32 [H * 128] or [H, 128], contiguous (raw gate mode).

  • lower_bound (float, optional) – The gate’s lower bound, required in raw gate mode: finite, -5.0 <= lower_bound < 0.0.

  • use_qk_l2norm_in_kernel (bool) – False (the default): q / k are L2-normalized by the caller and contiguous. True: q / k are the raw rows, normalized per token and head inside the kernels (bf16(x * r), r = 1 / sqrt(sum(x^2) + 1e-6) in FP32); they may be strided views (unit innermost stride, head stride 128, token stride a multiple of 8 elements and at least 128 * H, a 16-byte aligned address; never copied). The saved set adds the FP32 reciprocal norms [2, S, H] (no normalized rows are kept) and records the mode; the workspace must be allocated with use_qk_l2norm_in_kernel=True.

Returns:

(out, saved): out BF16 like v; saved a CakeKDADetChunkSaved (versioned) or None.

Return type:

tuple[torch.Tensor, Optional[Any]]