flashinfer.gemm.mm_bf16_fp8_workspace_bytes

flashinfer.gemm.mm_bf16_fp8_workspace_bytes(m: int, n: int, k: int, device: device | None = None, tactic: str | None = None) → int

Workspace bytes mm_bf16_fp8() needs for an [M, K] x [N, K] problem.

Parameters:
  • m (int) – The problem: m is the flattened row count of a (the product of its leading dimensions), n the weight’s row count, k the reduction depth. Same bounds as the call: m >= 0, n and k positive, k a multiple of 16, n a multiple of 8, each at most 2**31 - 512, and ceil(m / 64) * ceil(n / 64) at most 2**28 - 1.

  • n (int) – The problem: m is the flattened row count of a (the product of its leading dimensions), n the weight’s row count, k the reduction depth. Same bounds as the call: m >= 0, n and k positive, k a multiple of 16, n a multiple of 8, each at most 2**31 - 512, and ceil(m / 64) * ceil(n / 64) at most 2**28 - 1.

  • k (int) – The problem: m is the flattened row count of a (the product of its leading dimensions), n the weight’s row count, k the reduction depth. Same bounds as the call: m >= 0, n and k positive, k a multiple of 16, n a multiple of 8, each at most 2**31 - 512, and ceil(m / 64) * ceil(n / 64) at most 2**28 - 1.

  • device (Optional[torch.device]) – The CUDA device the call will run on (current device when omitted or given without an index). The answer depends on it.

  • tactic (Optional[str]) – The tactic the call will be made with (None = the launcher’s choice). The answer depends on it exactly as the launch does, so query with the arguments you will call with; a caller serving several shapes from one buffer takes the maximum over them.

Returns:

0 when the call needs no workspace (every m == 0 call, and every shape whose variant runs in a single pass), otherwise the byte count a workspace passed to mm_bf16_fp8() must reach.

Return type:

int

Raises:
  • ValueError – A shape outside the bounds above, an unknown tactic, a non-CUDA device, or a split-K tactic on a k that cannot be sliced (for m > 0).

  • BackendSupportedError – A device the kernels do not support (anything but compute capability 9.0, 10.0 or 10.3), as the call raises.