flashinfer.mla.dsv41_fp4_quantize_pack_sparse_mla_cache¶
- flashinfer.mla.dsv41_fp4_quantize_pack_sparse_mla_cache(latent_kv: Tensor, *, kv_layout: str = 'HND', inline_scale: bool = False) Tensor¶
Quantize complete DeepSeek-V4.1 latent-KV pages to the V41_FP4 ABI.
- Parameters:
latent_kv (torch.Tensor) – Contiguous CUDA BF16/FP16 tensor with shape
[num_pages, page_size, 512]. A singleton latent-head axis is also accepted in HND or NHD position. All 512 values per token (RoPE dims included) are quantized in groups of 16 to packed E2M1 with E4M3 scales (scale = amax/6), mirroring the FlashMLA V41_FP4 trajectory.kv_layout (str) – Output layout, either
"HND"or"NHD".inline_scale (bool) – Write inline-scale records (FlashMLA’s V4.1 ABI as of its 2026.09.30 release;
kv_cache_format="fp8_dsv41_inline_fp4_ca"/SparseMLASm120Wrapper(kv_scale_format="ue8m0_g32_inline")) instead of footer-scale pages: each token’s 256 data bytes followed by its own 32 scale bytes.
- Returns:
Opaque uint8 paged cache with logical shape
[num_pages, 1, page_size, 288]for HND or[num_pages, page_size, 1, 288]for NHD. Within each physical page it storespage_size * 256data bytes followed bypage_size * 32scale bytes. Consumers must not interpret the last dimension as a contiguous per-token record. Withinline_scale=Truethe last dimension is each token’s contiguous 288-byte record instead.- Return type:
torch.Tensor