flashinfer.cake_kda_det_train.cake_kda_det_train_backward¶
- flashinfer.cake_kda_det_train.cake_kda_det_train_backward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, dout: Tensor, *, saved: Any | None = None, scale: float | None = None, workspace: Any | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False) tuple[Tensor, ...]¶
Packed KDA chunked training backward.
- Parameters:
q – The forward’s operands.
k – The forward’s operands.
v – The forward’s operands.
g – The forward’s operands.
beta – The forward’s operands.
cu_seqlens – The forward’s operands.
dout (torch.Tensor) – BF16 gradient of
out, laid out likev.saved (CakeKDADetChunkSaved, optional) – The forward’s saved set (version 4 or 2);
Nonerecomputes it from the inputs (checkpoint replay).scale (float, optional) – The forward’s scale;
128 ** -0.5whenNone.workspace (CakeKDADetWorkspace, optional) – A reused workspace with backward buffers; without one this call allocates its own.
arena (CakeKDADetArena, optional) – The lower-memory arena option (
cake_kda_det_train_arena(); off by default) of that per-call workspace: the lean backward layouts and the tail split (tail_passes), the same gradients bitwise;Nonetakes a given workspace’s option, a differing one is refused. A lean arena serves no checkpoint replay (saved=None); a batched option (batch_tokens) runs the backward once per document group.doc_lengths (sequence of int, optional) – The documents’ token counts in
cu_seqlensorder (host integers), required under a batched option and checked againstcu_seqlenson the device.use_gate_in_kernel – The forward’s gate mode and raw gate parameters (see
cake_kda_det_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.A_log – The forward’s gate mode and raw gate parameters (see
cake_kda_det_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.dt_bias – The forward’s gate mode and raw gate parameters (see
cake_kda_det_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.lower_bound – The forward’s gate mode and raw gate parameters (see
cake_kda_det_train_forward()); a saved set of the other mode, or with anotherlower_bound, is refused.use_qk_l2norm_in_kernel (bool) – The forward’s in-kernel Q/K L2 norm mode (
q/kthen the raw rows;dq/dktheir gradients); a saved set of the other mode is refused.
- Returns:
(dq, dk, dv, dbeta, dg): BF16 gradients ofq, k, v, betaand the FP32 gradient ofg, ranked like the inputs. Withuse_gate_in_kernel=True:(dq, dk, dv, dbeta, dg, dA_log, ddt_bias),dgthe BF16 gradient of the raw gate (contiguous, ranked likeg),dA_logFP32[H]andddt_biasFP32 shaped likedt_bias. Bitwise reproducible for the same inputs.- Return type:
tuple of torch.Tensor