flashinfer.cake_kda_nondet_train.cake_kda_nondet_train_backward

flashinfer.cake_kda_nondet_train.cake_kda_nondet_train_backward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, cu_seqlens: Tensor, dout: Tensor, *, saved: Any | None = None, scale: float | None = None, workspace: Any | None = None, arena: Any | None = None, doc_lengths: Sequence[int] | None = None, use_gate_in_kernel: bool = False, A_log: Tensor | None = None, dt_bias: Tensor | None = None, lower_bound: float | None = None, use_qk_l2norm_in_kernel: bool = False) → tuple[Tensor, ...]

Packed KDA chunked training backward.

Parameters:
  • q – The forward’s operands.

  • k – The forward’s operands.

  • v – The forward’s operands.

  • g – The forward’s operands.

  • beta – The forward’s operands.

  • cu_seqlens – The forward’s operands.

  • dout (torch.Tensor) – BF16 gradient of out, laid out like v.

  • saved (CakeKDANondetChunkSaved, optional) – The forward’s saved set (version 4 or 2) of cake_kda_nondet_train_forward(); None recomputes it from the inputs (checkpoint replay). A saved set of the deterministic package is refused (ValueError).

  • scale (float, optional) – The forward’s scale; 128 ** -0.5 when None.

  • workspace (CakeKDANondetWorkspace, optional) – A reused workspace with backward buffers; without one this call allocates its own. A workspace of the deterministic package is refused (ValueError).

  • arena (CakeKDANondetArena, optional) – The lower-memory arena option (cake_kda_nondet_train_arena(); off by default) of that per-call workspace: the lean backward layouts and the tail split (tail_passes), the same gradients bitwise; None takes a given workspace’s option, a differing one is refused, as is an option of the deterministic package. A lean arena serves no checkpoint replay (saved=None); a batched option (batch_tokens) runs the backward once per document group.

  • doc_lengths (sequence of int, optional) – The documents’ token counts in cu_seqlens order (host integers), required under a batched option and checked against cu_seqlens on the device.

  • use_gate_in_kernel – The forward’s gate mode and raw gate parameters (see cake_kda_nondet_train_forward()); a saved set of the other mode, or with another lower_bound, is refused.

  • A_log – The forward’s gate mode and raw gate parameters (see cake_kda_nondet_train_forward()); a saved set of the other mode, or with another lower_bound, is refused.

  • dt_bias – The forward’s gate mode and raw gate parameters (see cake_kda_nondet_train_forward()); a saved set of the other mode, or with another lower_bound, is refused.

  • lower_bound – The forward’s gate mode and raw gate parameters (see cake_kda_nondet_train_forward()); a saved set of the other mode, or with another lower_bound, is refused.

  • use_qk_l2norm_in_kernel (bool) – The forward’s in-kernel Q/K L2 norm mode (q / k then the raw rows; dq / dk their gradients); a saved set of the other mode is refused.

Returns:

(dq, dk, dv, dbeta, dg): BF16 gradients of q, k, v, beta and the FP32 gradient of g, ranked like the inputs. With use_gate_in_kernel=True: (dq, dk, dv, dbeta, dg, dA_log, ddt_bias), dg the BF16 gradient of the raw gate (contiguous, ranked like g), dA_log FP32 [H] and ddt_bias FP32 shaped like dt_bias. Bitwise reproducible for the same inputs and path. dq, dk, dv and dbeta are bitwise the deterministic package’s and dg differs from it by FP32 rounding; in raw gate mode and with the in-kernel Q/K L2 norm every gradient is bitwise the deterministic package’s.

Return type:

tuple of torch.Tensor