flashinfer.fused_moe.QuantConfig

class flashinfer.fused_moe.QuantConfig(weight: QuantFormat = QuantFormat.BF16, activation: QuantFormat = QuantFormat.BF16, output: QuantFormat = QuantFormat.BF16, *, swizzled_scale_factors: bool | None = None, per_token_scale: bool | None = None)

Quantization scheme: MMA weight/activation formats plus the result format.

QuantConfig only checks that each axis is a QuantFormat. Legal combinations are runner capabilities (supported_quant_variants and supported_output_formats), not an allow-list in this dataclass.

Parameters:

  • weight (QuantFormat) – MMA weight and activation formats. An omitted axis is BF16, i.e. unquantized: QuantConfig() is BF16×BF16 and QuantConfig(weight=MXFP4) is MXFP4 weights with BF16 activations (W4A16). MXFP4×MXFP8 must be spelled with both axes.

  • activation (QuantFormat) – MMA weight and activation formats. An omitted axis is BF16, i.e. unquantized: QuantConfig() is BF16×BF16 and QuantConfig(weight=MXFP4) is MXFP4 weights with BF16 activations (W4A16). MXFP4×MXFP8 must be spelled with both axes.

  • output (QuantFormat) – Layer output format. Default BF16. Pass QuantFormat.FP16, not torch.float16.

  • swizzled_scale_factors (bool or None) – Whether block scale factors use the swizzled (vs linear) layout. None → backend default. Mirrors core’s swizzled_input_sf. Finer SfLayout (128x4 / 8x4 / linear) selection is deferred (design doc C42): unlike RoutingMethodType / ActivationType, SfLayout has no eval-safe __repr__, so exposing it here would break the eval(repr(cfg)) round-trip — a bool keeps this config serializable.

  • per_token_scale (bool or None) – Whether activations carry a per-token scale (vs per-tensor / block). None → backend default.

__init__(weight: QuantFormat = QuantFormat.BF16, activation: QuantFormat = QuantFormat.BF16, output: QuantFormat = QuantFormat.BF16, *, swizzled_scale_factors: bool | None = None, per_token_scale: bool | None = None) → None

Methods

__init__([weight, activation, output, ...])

Attributes

activation

output

pair

MMA (weight, activation) pair used for runner matching.

per_token_scale

swizzled_scale_factors

weight