flashinfer.fused_moe.CudnnGroupedGemmNvfp4Runner

class flashinfer.fused_moe.CudnnGroupedGemmNvfp4Runner(config: MoEConfig, device: device)

NVFP4 MoE over grouped_mm_fp4 (16-wide blocks).

Packed E2M1 uint8 [M, H // 2] rows with float8_e4m3fn [M, H // 16] token-major scales and a global scale of one; packed E2M1 weights with E4M3 block scales and one global dequant per expert. The intermediate is requantized with a global scale of one. Extra packed inputs: [fc1_weight_scale, fc2_weight_scale, fc1_dequant, fc2_dequant, hidden_states_scale].

__init__(config: MoEConfig, device: device) → None

Methods

__init__(config, device)
accepts(act, weights)	Select candidates with prepared weights and supported input geometry.
build()
check_support()
forward(inputs[, tactic, do_preparation])	Forward pass for tunable runners.
get_cache_key_extras(inputs)	Return extra values to include in the autotune cache key.
get_valid_tactics(inputs, _profile)	Rank each GEMM's plans on its own (the other GEMM at -1), then pair the top _num_top_tactics_per_stage of each and add the all-heuristic fallback.
launch_kwargs_for(inputs)	Return only the launch kwargs required by this packed call.
launch_state_for(inputs)	Return immutable per-call launch metadata paired with inputs.
pack_inputs(act, weights)
precompile_tactics(inputs, tactics, profile, ...)	Optionally perform compile-only work for all tactics before profiling.
supports_quant(quant)	Return whether this runner can execute quant's three format axes.
tuning_config_for(inputs)	Return the tuning config paired with inputs when one is present.

Attributes

backend_key
supported_activation_classes
supported_activation_classes_by_quant
supported_output_formats
supported_quant_variants
supported_routing_modes
supports_expert_parallelism
supports_fused_shared_experts
supports_nvfp4_4over6
supports_per_token_scale
weight_view_aliases
config