flashinfer.page

Kernels to manipulate paged kv-cache.

Append new K/V tensors to Paged KV-Cache

append_paged_kv_cache(append_key, ...[, ...])	Append a batch of key-value pairs to a paged key-value cache.
append_paged_mla_kv_cache(append_ckv, ...)	Append a batch of key-value pairs to a paged key-value cache, Note: current only support ckv=512 and kpe=64
nvfp4_quantize_append_paged_kv_cache(...[, ...])	Quantize and append K/V rows into an NVFP4 paged KV cache.
nvfp4_quantize_append_paged_kv_cache_with_slot_mapping(...)	Quantize and write K/V rows into an NVFP4 paged KV cache by slot mapping.
get_batch_indices_positions(append_indptr, ...)	Convert append indptr and sequence lengths to batch indices and positions.