flashinfer.fused_moe.CakeWarpDecodeRunner

class flashinfer.fused_moe.CakeWarpDecodeRunner(config: MoEConfig, device: device)

Exact-SM103 Cake runner for two calibrated NVFP4 decode geometries.

The runner consumes the physical tensor view produced by TrtllmFp4Config: packed E2M1 weights and activations, E4M3 block scales, and per-expert FP32 epilogue scales. The generated kernel fixes SwiGLU alpha=1 and beta=0; the compatible gemm1_alpha field in the TRTLLM view is therefore not a launch argument.

__init__(config: MoEConfig, device: device)

Methods

__init__(config, device)
build()
check_support()
forward(inputs[, tactic, do_preparation])	Forward pass for tunable runners.
get_cache_key_extras(inputs)	Return extra values to include in the autotune cache key.
get_valid_tactics(inputs, profile)	One tactic corresponding to one cuda kernel normally, but how to interpret the meaning of tactic is pure internal details of the runner.
launch_kwargs_for(inputs)	Return only the launch kwargs required by this packed call.
launch_state_for(inputs)	Return immutable per-call launch metadata paired with inputs.
pack_inputs(act, weights)	Validate and flatten the exact warp-decode TVM-FFI input ABI.
tuning_config_for(inputs)	Return the tuning config paired with inputs when one is present.

Attributes

backend_key
supported_activation_classes
supported_activation_classes_by_quant
supported_quant_variants
supported_routing_modes
supports_expert_parallelism
supports_fused_shared_experts
config