flashinfer.gemm.mm_fp16_fp8

flashinfer.gemm.mm_fp16_fp8(a: Tensor, b: Tensor, alpha: Tensor, *, bias: Tensor | None = None, out_dtype: dtype | None = None, out: Tensor | None = None, backend: Literal['auto', 'cute-dsl'] = 'auto', tactic: str | None = None, workspace: Tensor | None = None) → Tensor

float16 GEMM whose [N, K] operand is stored as FP8 e4m3 (W8A16).

out = a @ (b.float() * alpha).T + bias with the weight dequantized inside the kernel: mm_bf16_fp8() for float16 activations. Same kernels, same arguments, same contract, with float16 wherever that function says bfloat16: a is (..., K) float16; bias and out are float16; out_dtype accepts only torch.float16; the result is (..., N) in float16. The e4m3 weight is converted to float16 exactly, the products are accumulated in fp32, and alpha and bias are applied before the single rounding into float16. tactic takes the same keys (MM_BF16_FP8_TACTICS_SM100 / MM_BF16_FP8_TACTICS_SM90); mm_fp16_fp8_workspace_bytes() sizes the workspace. See mm_bf16_fp8() for the full description of every argument, the shape, alignment and aliasing rules, and the errors.