flashinfer.cake_gqa_det_train.cake_gqa_det_train_workspace_bytes¶
- flashinfer.cake_gqa_det_train.cake_gqa_det_train_workspace_bytes(total_q: int, num_heads_q: int, num_segments: int, *, total_k: int | None = None, num_heads_kv: int | None = None, max_seqlen_k: int | None = None, row_map: bool = False, budget_bytes: int | None = None, cu_seqlens_q_host: Sequence[int] | Tensor | None = None, cu_seqlens_k_host: Sequence[int] | Tensor | None = None) int¶
Bytes of backward workspace one call needs.
- Parameters:
total_q (int) –
T_q, the packed query rows.num_heads_q (int) –
H_q, the query heads.num_segments (int) –
B, the segments (len(cu_seqlens_q) - 1).total_k (int, optional) –
T_k, given together withnum_heads_kv; with both the dS^T workspace is included. One without the other raisesValueError.num_heads_kv (int, optional) –
H_kv, given together withtotal_k.max_seqlen_k (int, optional) – As in
cake_gqa_det().row_map (bool) – Include the two FP32
[T_k, H_kv, 256]window gradients of a row-map call (needstotal_kandnum_heads_kv, elseValueError).budget_bytes (int, optional) – As in
cake_gqa_det_train_workspace().cu_seqlens_q_host (sequence of int or torch.Tensor, optional) – As in
cake_gqa_det(): the exact dS^T chunk of the backward call that gets them.cu_seqlens_k_host (sequence of int or torch.Tensor, optional) – As in
cake_gqa_det(): the exact dS^T chunk of the backward call that gets them.
- Returns:
The bytes of FP32
[H_q / 2, rows_pad, 256]plus two[H_q / 2, rows_pad], withrows_pad = round_up(2 T_q + 128 B, 128), plus the dS^T chunk and the window gradients when asked.- Return type:
int