flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace_bytes¶
- flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace_bytes(total_q: int, num_heads_q: int, num_segments: int, *, total_k: int = 0, num_heads_kv: int = 0, max_seqlen_k: int | None = None, row_map: bool = False, budget_bytes: int | None = None, cu_seqlens_q_host: Sequence[int] | Tensor | None = None, cu_seqlens_k_host: Sequence[int] | Tensor | None = None) int¶
Bytes of device memory one backward call needs beside its operands and outputs.
- Parameters:
total_q (int) –
T_q, the packed query rows.num_heads_q (int) –
H_q, the query heads.num_segments (int) –
B, the segments (len(cu_seqlens_q) - 1).total_k (int, optional) –
T_k; withnum_heads_kvalso given the per-call dS^T workspace is included.num_heads_kv (int, optional) –
H_kv, given together withtotal_k.max_seqlen_k (int, optional) – As in
cake_gqa_nondet().row_map (bool) – Include the two per-call FP32
[T_k, H_kv, 256]window gradients of a row-map call.budget_bytes (int, optional) – As in
cake_gqa_nondet_train_workspace().cu_seqlens_q_host (sequence of int or torch.Tensor, optional) – As in
cake_gqa_nondet(): the exact dS^T chunk of the backward call that gets them.cu_seqlens_k_host (sequence of int or torch.Tensor, optional) – As in
cake_gqa_nondet(): the exact dS^T chunk of the backward call that gets them.
- Returns:
The bytes of FP32
[H_q / 2, rows_pad, 256]plus two[H_q / 2, rows_pad](the scratch a workspace keeps), withrows_pad = round_up(2 T_q + 128 B, 128), plus the dS^T chunk and the window gradients when asked.- Return type:
int