flashinfer.cute_dsl.attention.wrappers.batch_hca.cute_dsl_hca_decode¶
- flashinfer.cute_dsl.attention.wrappers.batch_hca.cute_dsl_hca_decode(query: Tensor, window_kv_cache: Tensor, compressed_kv_cache: Tensor, workspace_buffer: Tensor, window_indices: Tensor, compressed_block_tables: Tensor, hca_seq_lens: Tensor, sparse_topk_lens: Tensor, window_valid_lens: Tensor, softmax_scale: float, output_scale: float = 1.0, sinks: Tensor | None = None, out: Tensor | None = None, is_causal: bool = True, is_persistent: bool = False) Tensor¶
Run the FP8 DeepSeek V4 HCA decode kernel on SM100/SM103.
All tensors use PyTorch-native layouts.
queryis[B, Q, H, 512]; the window pool is flat[pool_rows, 512]with absolutewindow_indices[B * Q, 128]; and the compressed pool remains[num_pages, page_size, 512]with a page table. The current DSV4 integration is causal-only.hca_seq_lenscounts HCA slots (128 window slots plus compressed slots), not original uncompressed tokens.