flashinfer.fused_moe.MoEWeightPack

class flashinfer.fused_moe.MoEWeightPack(native_views: ~typing.Dict[str, ~typing.Dict[str, ~torch.Tensor]] = <factory>)

Long-lived weight container with per-backend native materializations.

Each backend’s native weight layout (quantized, swizzled, MMA-ordered, etc.) is stored under its backend_key. Populated once at model-load / layer-init via prepare_for(key, view); read on every call via get_view(key).

Holding multiple materializations is intentional — that’s the memory cost the user pays for cross-backend autotune. Each view is the exact kwargs dict that runner’s forward expects for weight-side arguments.

Every XxxConfig.prepare_weights takes the same canonical source layout: w1 is [E, 2I, H] for gated activations with rows in [up, gate] order (act(x @ w1[I:].T) * (x @ w1[:I].T)), or [E, I, H] for non-gated ones; w2 is [E, H, I]. Backends that want [gate, up] swap halves inside their prepare helper.

__init__(native_views: ~typing.Dict[str, ~typing.Dict[str, ~torch.Tensor]] = <factory>) → None

Methods

__init__([native_views])
get_view(backend_key)
prepare_for(backend_key, view)	Register a backend-native weight view.

Attributes

native_views