flashinfer.mla.dsv41_fp8_quantize_append_sparse_mla_cache

flashinfer.mla.dsv41_fp8_quantize_append_sparse_mla_cache(latent_kv: Tensor, slot_mapping: Tensor, cache: Tensor, *, inline_scale: bool = False) → None

Quantize and append DeepSeek-V4.1 latent KV by physical slot into a 528-byte FP8 cache.

Parameters:

  • latent_kv (torch.Tensor) – Contiguous CUDA BF16/FP16 tensor with one 512-element latent-KV row per entry in slot_mapping.

  • slot_mapping (torch.Tensor) – Contiguous 1D CUDA int32 or int64 tensor. slot_mapping[i] is page_id * page_size + entry_id. Negative and out-of-range slots are padding and are ignored. If a valid slot occurs more than once, the lowest-index input row is written deterministically.

  • cache (torch.Tensor) – Destination opaque uint8 paged cache (2D [num_pages, page_bytes] with page_bytes a multiple of 528, 3D [num_pages, page_size, 528], or the 4D HND/NHD forms). Page-strided views may place the cache inside vLLM’s packed physical block allocation (16-byte multiple page stride; rows inside a page stay packed). Only addressed data rows and scale slots are written, so prefill history is reused directly by decode. The conversion is the one documented for dsv41_fp8_quantize_pack_sparse_mla_cache().

  • inline_scale (bool) – Write inline-scale records (FlashMLA’s V4.1 ABI as of its 2026.09.30 release; kv_cache_format="fp8_dsv41_inline" / SparseMLASm120Wrapper(kv_scale_format="ue8m0_g32_inline")) instead of footer-scale pages: each token’s 512 data bytes followed by its own 16 scale bytes. slot_mapping then addresses row page_id * (page_stride // row_pitch) + entry_id of the cache’s records, at the cache’s row pitch (a 16-byte multiple of at least 528 bytes).