flashinfer.mla.SparseMLASm120DecodeConfig

class flashinfer.mla.SparseMLASm120DecodeConfig(d_qk: int, page_block_size: int, max_num_tokens: int, topks: frozenset[int], min_topk: int, max_num_heads: int)

Instantiated decode-kernel set for one SM120 sparse-MLA kernel family.

Decode-form calls (num_tokens <= max_num_tokens) prefer a standalone decode kernel when their shape matches one of the instantiations described here; decode-eligible is not required, since the prefill orchestrator serves any remaining decode-form shape at num_tokens >= 1 (and crossover calibration may route even eligible shapes to prefill past a measured num_tokens threshold). Larger calls go through the prefill orchestrator, which has its own separately instantiated shape envelope; this config describes decode only.

d_qk

Query/key head dim served by this family (512 for DSv4 / GLM53_NOPE, 576 for DSv3.2 / GLM-NSA, 1088 for the DOTS3_SWA sliding-window family, whose d_v is then 1024).

Type:

int

page_block_size

The only KV page block size the decode kernels are instantiated for.

Type:

int

max_num_tokens

Largest num_tokens routed to the decode kernels (inclusive).

Type:

int

topks

The calibrated top-k values (the crossover sweep points). Decode serves ANY topk >= min_topk — topk is a runtime kernel argument — so this set is documentation of what has measured crossover data, not the eligibility boundary.

Type:

frozenset[int]

min_topk

Smallest legal topk (the indices-row width). 513 for the sliding-window family (the window must fit the buffer); 1 elsewhere.

Type:

int

max_num_heads

Every num_heads in [1, max_num_heads] is served: dedicated instantiations at {8, 16, 32, 64, 128} plus one runtime-head-count instantiation covering any other count.

Type:

int

__init__(d_qk: int, page_block_size: int, max_num_tokens: int, topks: frozenset[int], min_topk: int, max_num_heads: int) → None

Methods

__init__(d_qk, page_block_size, ...)
supported_num_heads()	Every head count from 1 through max_num_heads (runtime-H).
supported_topk([num_heads])	Sorted calibrated top-k values for num_heads (or any head count).
supports_decode(num_heads, topk, *[, ...])	True iff a decode-form call with this shape is decode-instantiated.

Attributes

d_qk
page_block_size
max_num_tokens
topks
min_topk
max_num_heads