flashinfer.fused_moe.MoEFinalizeConfig

class flashinfer.fused_moe.MoEFinalizeConfig(do_finalize: bool = True, use_fused_finalize: bool = True)

How the finalize (combine) step behaves.

Split out of ExecutionConfig for the same reason RoutingConfig is its own config: finalize is a distinct architectural concern (how the per-expert partials are reduced back into one row per token), not a runtime knob like PDL or the autotuner token budget.

Parameters:

• do_finalize (bool) – Whether to apply routing-weight scaling and accumulate the per-expert partial results into the output. False returns the unreduced TRTLLM intermediates as [gemm2_output, expert_weights, expanded_idx_to_permuted_idx], leaving the combine to the caller. For FromLogits routing, the routing kernel emits expert_weights in bfloat16 regardless of the routing-logits dtype. PackedPrecomputed routing also yields bfloat16 weights: the caller’s values are narrowed to bfloat16 when packed into the top-k ids. Only UnpackedPrecomputed routing preserves the caller-provided weights dtype, since it forwards topk_weights to the kernel unchanged. Only backends that advertise unfinalized output support this mode.

• use_fused_finalize (bool) – Whether supported backends reduce routed outputs in the GEMM2 epilogue (atomic accumulation) instead of running a separate reduction kernel. Backends that do not support it ignore the flag.

__init__(do_finalize: bool = True, use_fused_finalize: bool = True) → None

Methods

__init__([do_finalize, use_fused_finalize])

Attributes

do_finalize
use_fused_finalize