flashinfer.sdsa.token_sparse.union_key_limit

flashinfer.sdsa.token_sparse.union_key_limit(group: int, top_k: int, device=None) → int

The most keys group_union_masks() takes for groups of group rows and top_k keys each: its key bitmap and mask rows must fit in shared memory.

>>> union_key_limit(4, 2048)  # 227 KiB of shared memory, as on SM100
1441792
>>> union_key_limit(8, 2048)
1310720
>>> union_key_limit(32, 2048)
1179648
>>> union_key_limit(32, 512)
1310720