flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_varlen¶
- flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_varlen(q_latent: Tensor, q_rope: Tensor, kv_latent: Tensor, k_rope: Tensor, gather_kv_indices: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, max_seqlen_q: int | None = None, max_seqlen_k: int | None = None, *, causal: bool = False, topk_length: Tensor | None = None, softmax_scale: float | None = None, return_lse: bool = False, key_passes: int | None = None, backend: str = 'cake', dkv_acc: Tensor | Callable | None = None, dkv_dst_map: Tensor | None = None, dkv_direct: bool | None = None, check_dst_map: bool | None = None)¶
Packed multi-document form of
cake_dsa_sparse_attention_det()(SM100 / SM103 / SM107); every output repeats bit for bit.Documents are packed along the first dimension of the query tensors (
T = cu_seqlens_q[-1]) and of the key tensors (S = cu_seqlens_k[-1]);cu_seqlens_qandcu_seqlens_kare independent and query and key lengths may differ per document: the query segment is the tail of its key prefix, so querylocal_qof documentdsits at key position(seqlen_k[d] - seqlen_q[d]) + local_q(a context-parallel tail or a GLM-style packed batch).gather_kv_indices[T, topk]int32 hold key positions relative to the row’s document (-1or>= seqlen_k[d]invalid) and are offset bycu_seqlens_kon device in one launch right before the forward kernel; each row is compacted (valid keys first, in slot order, then-1) so rows with holes cost their valid slots, and a callertopk_lengthis a position cut applied before compaction. Zero-length query segments contribute no rows; a row whose every slot is invalid givesout = 0,lse = -infand zero gradients. The kernels apply no positional mask of their own; the offset index rows define the key sets.max_seqlen_q/max_seqlen_kare accepted for signature parity with other varlen attention entry points and are not read on the host. Seecake_dsa_sparse_attention_det()for the other arguments and the returns; the strided query slices and packed key rows described there are accepted here too.- Parameters:
causal (bool) – With
False(default) the index row is taken as is – only-1/ out-of-range slots are dropped. WithTruea slot that selects a key after the query’s own position (idx > (seqlen_k[d] - seqlen_q[d]) + local_q) is invalid too, so a top-k selector need not enforce causality itself.key_passes (Optional[int]) – As in
cake_dsa_sparse_attention_det(): the key-range passes of the backward’s main stage (None: the key-pass policy chooses), a scheduling choice only; the gradients are bit for bit those of one pass.dkv_direct (Optional[bool]) – As in
cake_dsa_sparse_attention_det(): accepted for parity, validated, without effect.check_dst_map (Optional[bool]) – As in
cake_dsa_sparse_attention_det():None(default) readsFLASHINFER_CAKE_DSA_CHECK_DST_MAP(unset, empty or0= no check);Truevalidates the values ofdkv_dst_mapon every backward,Falsenever does.