flashinfer.sdsa.token_sparse.union_key_limit¶
- flashinfer.sdsa.token_sparse.union_key_limit(group: int, top_k: int, device=None) int¶
The most keys
group_union_masks()takes for groups ofgrouprows andtop_kkeys each: its key bitmap and mask rows must fit in shared memory.>>> union_key_limit(4, 2048) # 227 KiB of shared memory, as on SM100 1441792 >>> union_key_limit(8, 2048) 1310720 >>> union_key_limit(32, 2048) 1179648 >>> union_key_limit(32, 512) 1310720