flashinfer.cake_gqa_det_train.cake_gqa_det_train_backward

flashinfer.cake_gqa_det_train.cake_gqa_det_train_backward(q: Tensor, k: Tensor, v: Tensor, out: Tensor, lse: Tensor, dout: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, *, softmax_scale: float | None = None, dq: Tensor | None = None, dk: Tensor | None = None, dv: Tensor | None = None, dkv_dtype: dtype = torch.bfloat16, dk_acc: Tensor | None = None, dv_acc: Tensor | None = None, row_map: Tensor | None = None, max_seqlen_k: int | None = None, workspace: Any | None = None, cu_seqlens_q_host: Sequence[int] | Tensor | None = None, cu_seqlens_k_host: Sequence[int] | Tensor | None = None) → tuple[Tensor, Tensor | None, Tensor | None]

Backward without autograd: (dq, dk, dv) from the forward’s out and lse.

dK / dV are FP32 sums over the query heads of each key head.

Parameters:
  • q (torch.Tensor) – The forward’s operands (see cake_gqa_det()).

  • k (torch.Tensor) – The forward’s operands (see cake_gqa_det()).

  • v (torch.Tensor) – The forward’s operands (see cake_gqa_det()).

  • out (torch.Tensor) – The forward’s BF16 [T_q, H_q, 256] output.

  • lse (torch.Tensor) – The forward’s FP32 [H_q, T_q] LSE.

  • dout (torch.Tensor) – Contiguous BF16 [T_q, H_q, 256] gradient of out.

  • cu_seqlens_q (torch.Tensor) – As in cake_gqa_det().

  • cu_seqlens_k (torch.Tensor) – As in cake_gqa_det().

  • softmax_scale (float, optional) – The forward’s scale; default 256 ** -0.5.

  • dq (torch.Tensor, optional) – A contiguous BF16 [T_q, H_q, 256] to write; None allocates one.

  • dk (torch.Tensor, optional) – Contiguous [T_k, H_kv, 256] tensors of dkv_dtype to write (overwritten); None allocates them. Not used with row_map.

  • dv (torch.Tensor, optional) – Contiguous [T_k, H_kv, 256] tensors of dkv_dtype to write (overwritten); None allocates them. Not used with row_map.

  • dkv_dtype (torch.dtype) – torch.bfloat16 (default): dK / dV rounded once to BF16. torch.float32: the FP32 window gradients are stored.

  • dk_acc (torch.Tensor, optional) – FP32 [N_dst, H_kv, 256] destinations that the window gradients are added into; they need row_map.

  • dv_acc (torch.Tensor, optional) – FP32 [N_dst, H_kv, 256] destinations that the window gradients are added into; they need row_map.

  • row_map (torch.Tensor, optional) – int32 [T_k]: window row j adds into destination row row_map[j] (rows may repeat; out-of-range rows are not checked).

  • max_seqlen_k (int, optional) – As in cake_gqa_det().

  • workspace (optional) – As in cake_gqa_det().

  • cu_seqlens_q_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_det().

  • cu_seqlens_k_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_det().

Returns:

(dq, dk, dv); with row_map, dk / dv are dk_acc / dv_acc.

Return type:

tuple[torch.Tensor, Optional[torch.Tensor], Optional[torch.Tensor]]

Notes

Bitwise reproducible run to run for one shape, one max_seqlen_k, one workspace budget_bytes and the same host boundaries (given or not): dK and dV of a key block are summed in tensor memory in one order, dS^T goes to the workspace and dQ is a GEMM over it with each output tile summed in one order, and the row-map destinations are folded in ascending window order. A call whose dS^T exceeds the budget runs in chunks (KV heads first, then key-block ranges); the budget and the host boundaries move the dQ rounding, not its accuracy. Raises no PyTorch determinism alert.