flashinfer.mla.supported_sparse_mla_sm120_configs

flashinfer.mla.supported_sparse_mla_sm120_configs(*, kv_cache_format: str = 'fp8') → dict[str, SparseMLASm120DecodeConfig]

Enumerate the instantiated SM120 sparse-MLA decode kernel configurations.

Lets callers validate a serving configuration at initialization time instead of discovering an uninstantiated (num_heads, topk) pair on the first decode-form request.

Parameters:

kv_cache_format ({"fp8", "nvfp4"}) – Storage format whose independently calibrated kernel envelope is requested. Defaults to "fp8" for backward compatibility.

Returns:

Mapping from kernel family to its instantiated decode set. FP8 returns DSv4, DSv3.2, GLM-NSA, GLM53_NOPE, and DOTS3_SWA entries. NVFP4 currently returns the independently calibrated DSv4 entry.

Return type:

dict[str, SparseMLASm120DecodeConfig]

Examples

>>> import flashinfer
>>> configs = flashinfer.mla.supported_sparse_mla_sm120_configs()
>>> configs["dsv4"].supports_decode(num_heads=64, topk=256)
True
>>> nvfp4 = flashinfer.mla.supported_sparse_mla_sm120_configs(
...     kv_cache_format="nvfp4"
... )
>>> nvfp4["dsv4"].bytes_per_token
384