flashinfer.page

Kernels to manipulate paged kv-cache.

Append new K/V tensors to Paged KV-Cache

append_paged_kv_cache(append_key, ...[, ...])	Append a batch of key-value pairs to a paged key-value cache.
append_paged_mla_kv_cache(append_ckv, ...)	Append a batch of key-value pairs to a paged key-value cache, Note: current only support ckv=512 and kpe=64
get_batch_indices_positions(append_indptr, ...)	Convert append indptr and sequence lengths to batch indices and positions.