flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace_bytes

flashinfer.cake_gqa_nondet_train.cake_gqa_nondet_train_workspace_bytes(total_q: int, num_heads_q: int, num_segments: int, *, total_k: int = 0, num_heads_kv: int = 0, max_seqlen_k: int | None = None, row_map: bool = False, budget_bytes: int | None = None, cu_seqlens_q_host: Sequence[int] | Tensor | None = None, cu_seqlens_k_host: Sequence[int] | Tensor | None = None) → int

Bytes of device memory one backward call needs beside its operands and outputs.

Parameters:
  • total_q (int) – T_q, the packed query rows.

  • num_heads_q (int) – H_q, the query heads.

  • num_segments (int) – B, the segments (len(cu_seqlens_q) - 1).

  • total_k (int, optional) – T_k; with num_heads_kv also given the per-call dS^T workspace is included.

  • num_heads_kv (int, optional) – H_kv, given together with total_k.

  • max_seqlen_k (int, optional) – As in cake_gqa_nondet().

  • row_map (bool) – Include the two per-call FP32 [T_k, H_kv, 256] window gradients of a row-map call.

  • budget_bytes (int, optional) – As in cake_gqa_nondet_train_workspace().

  • cu_seqlens_q_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_nondet(): the exact dS^T chunk of the backward call that gets them.

  • cu_seqlens_k_host (sequence of int or torch.Tensor, optional) – As in cake_gqa_nondet(): the exact dS^T chunk of the backward call that gets them.

Returns:

The bytes of FP32 [H_q / 2, rows_pad, 256] plus two [H_q / 2, rows_pad] (the scratch a workspace keeps), with rows_pad = round_up(2 T_q + 128 B, 128), plus the dS^T chunk and the window gradients when asked.

Return type:

int