flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_backward

flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_backward(q: Tensor, k: Tensor, v: Tensor, out: Tensor, lse: Tensor, dout: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, *, softmax_scale: float | None = None, dq: Tensor | None = None, dk: Tensor | None = None, dv: Tensor | None = None, dkv_dtype: dtype = torch.bfloat16, dk_acc: Tensor | None = None, dv_acc: Tensor | None = None, row_map: Tensor | None = None, max_seqlen_k: int | None = None, workspace: Any | None = None, cu_seqlens_q_host: Sequence[int] | Tensor | None = None, cu_seqlens_k_host: Sequence[int] | Tensor | None = None) → tuple[Tensor, Tensor | None, Tensor | None]

Backward without autograd: (dq, dk, dv) from the forward’s out and lse.

dK / dV are FP32 sums over the query heads of each key head.

Parameters:
  • q (torch.Tensor) – The forward’s operands (see cake_gqa_nondet()).

  • k (torch.Tensor) – The forward’s operands (see cake_gqa_nondet()).

  • v (torch.Tensor) – The forward’s operands (see cake_gqa_nondet()).

  • out (torch.Tensor) – The forward’s BF16 [T_q, H_q, 256] output.

  • lse (torch.Tensor) – The forward’s FP32 [H_q, T_q] LSE.

  • dout (torch.Tensor) – Contiguous BF16 [T_q, H_q, 256] gradient of out.

  • cu_seqlens_q (torch.Tensor) – As in cake_gqa_nondet().

  • cu_seqlens_k (torch.Tensor) – As in cake_gqa_nondet().

  • softmax_scale (float, optional) – The forward’s scale; default 256 ** -0.5.

  • dq (torch.Tensor, optional) – A contiguous BF16 [T_q, H_q, 256] to write; None allocates one.

  • dk (torch.Tensor, optional) – Contiguous [T_k, H_kv, 256] tensors of dkv_dtype to write (overwritten); None allocates them. Not used with row_map.

  • dv (torch.Tensor, optional) – Contiguous [T_k, H_kv, 256] tensors of dkv_dtype to write (overwritten); None allocates them. Not used with row_map.

  • dkv_dtype (torch.dtype) – torch.bfloat16 (default): dK / dV rounded once to BF16. torch.float32: the FP32 window gradients are stored.

  • dk_acc (torch.Tensor, optional) – FP32 [N_dst, H_kv, 256] destinations that the window gradients are added into; they need row_map.

  • dv_acc (torch.Tensor, optional) – FP32 [N_dst, H_kv, 256] destinations that the window gradients are added into; they need row_map.

  • row_map (torch.Tensor, optional) – int32 [T_k]: window row j adds into destination row row_map[j] (rows may repeat; out-of-range rows are not checked); a destination row becomes ((dst + w_j1) + w_j2) + ... over its window rows in ascending j.

  • max_seqlen_k (int, optional) – As in cake_gqa_nondet().

  • workspace (optional) – As in cake_gqa_nondet().

  • cu_seqlens_q_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_nondet().

  • cu_seqlens_k_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_nondet().

Returns:

(dq, dk, dv); with row_map, dk / dv are dk_acc / dv_acc.

Return type:

tuple[torch.Tensor, Optional[torch.Tensor], Optional[torch.Tensor]]

Notes

Bitwise reproducible run to run: dS^T goes to a workspace chunk (allocated per call, at most the workspace’s budget_bytes above the minimum chunk), dK / dV are summed by the CTA pair that owns their key block, dQ is a GEMM over the workspace, and the row-map destinations are folded in window order; nothing adds with atomics. The bits depend on the shapes, max_seqlen_k, the budget and the host boundaries only (another budget or other host boundaries may move the last bits of dQ). No PyTorch determinism alert is raised.