flashinfer.cake_gqa_det_train.cake_gqa_det_train_forward¶
- flashinfer.cake_gqa_det_train.cake_gqa_det_train_forward(q: Tensor, k: Tensor, v: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, *, softmax_scale: float | None = None, out: Tensor | None = None, lse: Tensor | None = None, workspace: Any | None = None) tuple[Tensor, Tensor]¶
Forward without autograd, causal with bottom-right alignment per segment.
- Parameters:
q (torch.Tensor) – As in
cake_gqa_det().k (torch.Tensor) – As in
cake_gqa_det().v (torch.Tensor) – As in
cake_gqa_det().cu_seqlens_q (torch.Tensor) – As in
cake_gqa_det().cu_seqlens_k (torch.Tensor) – As in
cake_gqa_det().softmax_scale (float, optional) – Default
256 ** -0.5.out (torch.Tensor, optional) – A contiguous BF16
[T_q, H_q, 256]output to write;Noneallocates one.lse (torch.Tensor, optional) – A contiguous FP32
[H_q, T_q]natural-log LSE to write;Noneallocates one.workspace (optional) – As in
cake_gqa_det().
- Returns:
(out, lse). A row that sees no key getsout = 0andlse = -inf.- Return type:
tuple[torch.Tensor, torch.Tensor]
Notes
Fixed-order: it never raises the deterministic-algorithms alert.