flashinfer.fused_moe.preprocess_moe_weights_for_sm90_mixed_gemm_humming

flashinfer.fused_moe.preprocess_moe_weights_for_sm90_mixed_gemm_humming(weight: Tensor, raw_scale: Tensor, max_range: int = 11, *, interleave: bool = True) → Tuple[Tensor, Tensor, Tensor]

Prepare MXFP4 weights for the SM90 Humming-style FP8 activation path.

Parameters:

  • weight (torch.Tensor) – [num_experts, rows, K // 2] CUDA uint8 tensor containing packed MXFP4 payload values.

  • raw_scale (torch.Tensor) – [num_experts, rows, K // 32] CUDA uint8 tensor containing original E8M0 MXFP4 weight scales.

  • max_range (int) – Maximum per-expert E8M0 exponent range kept in the pre-MMA FP4->E4M3 offset. Humming uses 11 for FP8 activation.

  • interleave (bool) – If true, return tensors ready for cutlass_fused_moe. If false, return the logical processed weight and logical offset scale; this is useful for validation against a dequantized or Humming reference.

Returns:

(weight_out, scale_out, residual). With interleave=True, weight_out is the SM90 mixed-input weight layout and scale_out is the folded scale layout. With interleave=False, they are the logical processed packed weight and logical offset scale. residual is one FP32 value per expert and should be folded into the routed-token activation scale together with Humming’s fixed 2^6 compensation.

Return type:

Tuple[torch.Tensor, torch.Tensor, torch.Tensor]

Notes

The E8M0 range clamp, residual-scale factorization, and FP4 payload-rewrite scheme are adapted from Humming.