flashinfer.gemm.mm_bf16_fp8_workspace_bytes¶
- flashinfer.gemm.mm_bf16_fp8_workspace_bytes(m: int, n: int, k: int, device: device | None = None, tactic: str | None = None) int¶
Workspace bytes
mm_bf16_fp8()needs for an[M, K] x [N, K]problem.- Parameters:
m (int) – The problem:
mis the flattened row count ofa(the product of its leading dimensions),nthe weight’s row count,kthe reduction depth. Same bounds as the call:m >= 0,nandkpositive,ka multiple of 16,na multiple of 8, each at most2**31 - 512, andceil(m / 64) * ceil(n / 64)at most2**28 - 1.n (int) – The problem:
mis the flattened row count ofa(the product of its leading dimensions),nthe weight’s row count,kthe reduction depth. Same bounds as the call:m >= 0,nandkpositive,ka multiple of 16,na multiple of 8, each at most2**31 - 512, andceil(m / 64) * ceil(n / 64)at most2**28 - 1.k (int) – The problem:
mis the flattened row count ofa(the product of its leading dimensions),nthe weight’s row count,kthe reduction depth. Same bounds as the call:m >= 0,nandkpositive,ka multiple of 16,na multiple of 8, each at most2**31 - 512, andceil(m / 64) * ceil(n / 64)at most2**28 - 1.device (Optional[torch.device]) – The CUDA device the call will run on (current device when omitted or given without an index). The answer depends on it.
tactic (Optional[str]) – The
tacticthe call will be made with (None= the launcher’s choice). The answer depends on it exactly as the launch does, so query with the arguments you will call with; a caller serving several shapes from one buffer takes the maximum over them.
- Returns:
0when the call needs no workspace (everym == 0call, and every shape whose variant runs in a single pass), otherwise the byte count aworkspacepassed tomm_bf16_fp8()must reach.- Return type:
int
- Raises:
ValueError – A shape outside the bounds above, an unknown
tactic, a non-CUDA device, or a split-Ktacticon akthat cannot be sliced (form > 0).BackendSupportedError – A device the kernels do not support (anything but compute capability 9.0, 10.0 or 10.3), as the call raises.