flashinfer.cake_gqa_det_train.cake_gqa_det_train_forward

flashinfer.cake_gqa_det_train.cake_gqa_det_train_forward(q: Tensor, k: Tensor, v: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, *, softmax_scale: float | None = None, out: Tensor | None = None, lse: Tensor | None = None, workspace: Any | None = None) → tuple[Tensor, Tensor]

Forward without autograd, causal with bottom-right alignment per segment.

Parameters:
  • q (torch.Tensor) – As in cake_gqa_det().

  • k (torch.Tensor) – As in cake_gqa_det().

  • v (torch.Tensor) – As in cake_gqa_det().

  • cu_seqlens_q (torch.Tensor) – As in cake_gqa_det().

  • cu_seqlens_k (torch.Tensor) – As in cake_gqa_det().

  • softmax_scale (float, optional) – Default 256 ** -0.5.

  • out (torch.Tensor, optional) – A contiguous BF16 [T_q, H_q, 256] output to write; None allocates one.

  • lse (torch.Tensor, optional) – A contiguous FP32 [H_q, T_q] natural-log LSE to write; None allocates one.

  • workspace (optional) – As in cake_gqa_det().

Returns:

(out, lse). A row that sees no key gets out = 0 and lse = -inf.

Return type:

tuple[torch.Tensor, torch.Tensor]

Notes

Fixed-order: it never raises the deterministic-algorithms alert.