flashinfer.cake_kda_nondet_train.cake_kda_nondet_train_backward¶
- flashinfer.cake_kda_nondet_train.cake_kda_nondet_train_backward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, dout: Tensor, *, saved: Any | None = None, scale: float | None = None, workspace: Any | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False) tuple[Tensor, ...]¶
Packed KDA chunked training backward.
- Parameters:
q – The forward’s operands.
k – The forward’s operands.
v – The forward’s operands.
g – The forward’s operands.
beta – The forward’s operands.
cu_seqlens – The forward’s operands.
dout (torch.Tensor) – BF16 gradient of
out, laid out likev.saved (CakeKDANondetChunkSaved, optional) – The forward’s saved set (version 4 or 2) of
cake_kda_nondet_train_forward();Nonerecomputes it from the inputs (checkpoint replay). A saved set of the deterministic package is refused (ValueError).scale (float, optional) – The forward’s scale;
128 ** -0.5whenNone.workspace (CakeKDANondetWorkspace, optional) – A reused workspace with backward buffers; without one this call allocates its own. A workspace of the deterministic package is refused (
ValueError).arena (CakeKDANondetArena, optional) – The lower-memory arena option (
cake_kda_nondet_train_arena(); off by default) of that per-call workspace: the lean backward layouts and the tail split (tail_passes), the same gradients bitwise;Nonetakes a given workspace’s option, a differing one is refused, as is an option of the deterministic package. A lean arena serves no checkpoint replay (saved=None); a batched option (batch_tokens) runs the backward once per document group.doc_lengths (sequence of int, optional) – The documents’ token counts in
cu_seqlensorder (host integers), required under a batched option and checked againstcu_seqlenson the device.use_gate_in_kernel – The forward’s gate mode and raw gate parameters (see
cake_kda_nondet_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.A_log – The forward’s gate mode and raw gate parameters (see
cake_kda_nondet_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.dt_bias – The forward’s gate mode and raw gate parameters (see
cake_kda_nondet_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.lower_bound – The forward’s gate mode and raw gate parameters (see
cake_kda_nondet_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.use_qk_l2norm_in_kernel (bool) – The forward’s in-kernel Q/K L2 norm mode (
q/kthen the raw rows;dq/dktheir gradients); a saved set of the other mode is refused.
- Returns:
(dq, dk, dv, dbeta, dg): BF16 gradients ofq, k, v, betaand the FP32 gradient ofg, ranked like the inputs. Withuse_gate_in_kernel=True:(dq, dk, dv, dbeta, dg, dA_log, ddt_bias),dgthe BF16 gradient of the raw gate (contiguous, ranked likeg),dA_logFP32[H]andddt_biasFP32 shaped likedt_bias. Bitwise reproducible for the same inputs and path.dq,dk,dvanddbetaare bitwise the deterministic package’s anddgdiffers from it by FP32 rounding; in raw gate mode and with the in-kernel Q/K L2 norm every gradient is bitwise the deterministic package’s.- Return type:
tuple of torch.Tensor