flashinfer.cake_dsa_sparse_attention_nondet_train.cake_dsa_sparse_attention_nondet_varlen

flashinfer.cake_dsa_sparse_attention_nondet_train.cake_dsa_sparse_attention_nondet_varlen(q_latent: Tensor, q_rope: Tensor, kv_latent: Tensor, k_rope: Tensor, gather_kv_indices: Tensor, cu_seqlens_q: Tensor, cu_seqlens_k: Tensor, max_seqlen_q: int | None = None, max_seqlen_k: int | None = None, *, causal: bool = False, topk_length: Tensor | None = None, softmax_scale: float | None = None, return_lse: bool = False, key_passes: int | None = None, backend: str = 'cake', dkv_acc: Tensor | Callable | None = None, dkv_dst_map: Tensor | None = None, dkv_direct: bool | None = None, check_dst_map: bool | None = None)

Packed multi-document form of cake_dsa_sparse_attention_nondet() (SM100 / SM103 / SM107).

Documents are packed along the first dimension of the query tensors (T = cu_seqlens_q[-1]) and of the key tensors (S = cu_seqlens_k[-1]); cu_seqlens_q and cu_seqlens_k are independent and query and key lengths may differ per document: the query segment is the tail of its key prefix, so query local_q of document d sits at key position (seqlen_k[d] - seqlen_q[d]) + local_q (a context-parallel tail or a GLM-style packed batch). gather_kv_indices [T, topk] int32 hold key positions relative to the row’s document (-1 or >= seqlen_k[d] invalid) and are offset by cu_seqlens_k on device in one launch right before the forward kernel; each row is compacted (valid keys first, in slot order, then -1) so rows with holes cost their valid slots, and a caller topk_length is a position cut applied before compaction. Zero-length query segments contribute no rows; a row whose every slot is invalid gives out = 0, lse = -inf and zero gradients. The kernels apply no positional mask of their own; the offset index rows define the key sets. max_seqlen_q / max_seqlen_k are accepted for signature parity with other varlen attention entry points and are not read on the host. See cake_dsa_sparse_attention_nondet() for the other arguments and the returns; the strided query slices and packed key rows described there are accepted here too.

Under torch.use_deterministic_algorithms(True) the forward runs and the backward raises PyTorch’s RuntimeError before its first backward kernel launch (its dK / dV atomics are not run-to-run reproducible); deterministic preparation may run first (see the module docstring). With warn_only=True it issues a UserWarning and runs. No deterministic package is chosen instead.

Parameters:
  • causal (bool) – With False (default) the index row is taken as is – only -1 / out-of-range slots are dropped – exactly as in the first release. With True a slot that selects a key after the query’s own position (idx > (seqlen_k[d] - seqlen_q[d]) + local_q) is invalid too, so a top-k selector need not enforce causality itself (the rule GLM-style trainers with query segments that are tails of their key prefix rely on).

  • dkv_direct (Optional[bool]) – As in cake_dsa_sparse_attention_nondet(): None (default) reads FLASHINFER_CAKE_DSA_DKV_DIRECT (auto = the record’s size rule, 1 / 0 = force / disable the direct accumulation into dkv_acc); True / False decide explicitly.

  • check_dst_map (Optional[bool]) – As in cake_dsa_sparse_attention_nondet(): None (default) reads FLASHINFER_CAKE_DSA_CHECK_DST_MAP (unset, empty or 0 = no check); True validates the values of dkv_dst_map on every backward, False never does.