flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_nope¶
- flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_nope(q: Tensor, kv: Tensor, indices: Tensor, *, topk_length: Tensor | None = None, softmax_scale: float | None = None, return_lse: bool = False, key_passes: int | None = None, backend: str = 'cake', dkv_acc: Tensor | Callable | None = None, dkv_dst_map: Tensor | None = None, dkv_direct: bool | None = None, check_dst_map: bool | None = None)¶
Differentiable top-k sparse MLA attention without a rope part (NoPE: the whole score is the 512-dimensional latent product) for training, 64 query heads, SM100 / SM103 / SM107 only; every output repeats bit for bit.
O = softmax(softmax_scale * (q . kv^T)) . kvover the keys a row’sindicesselect (K = V = the shared latentkv). The native kernels of this form read no rope operands and produce one combined gradient per key row,dkv = dK + dV, natural layout, FP32.- Parameters:
q (torch.Tensor) – BF16
[T, 64, 512]absorbed query latents; contiguous or a strided token / head view (unit innermost stride, head and token strides positive multiples of 8 elements, head stride at least 512, 16-byte-aligned base), consumed in place.kv (torch.Tensor) – BF16
[S, 512]shared key/value latent, or its one-head form[S, 1, 512](squeezed to[S, 512]on the host; the gradient reaches the[S, 1, 512]leaf through that view).indices (torch.Tensor) – int32
[T, K_slots]global key rows intokv. AnyK_slots >= 1;-1or a value>= Smarks an invalid slot, anywhere in the row. A row whose every slot is invalid givesout = 0,lse = -infand zero gradients.topk_length (Optional[torch.Tensor]) – int32
[T]; slots>= topk_length[t]are invalid regardless of their content. Without it the route follows the table size (cake_backend.NOPE_CONVERT_SLOT_LIMITslots): below the limit the per-row lengths (last valid slot + 1) are derived by the forward kernel while it runs and saved for the backward with the rows as given; at or above it the rows are compacted on device first (valid keys first in slot order, then-1, the valid count as the length) and the compacted pair is saved. Hole-free rows give the same bytes on either route.softmax_scale (Optional[float]) – Defaults to
1 / 16.return_lse (bool) – Also return the natural-log logsumexp
[T, 64]FP32 of the scaled scores over the valid keys (-inffor fully masked rows).key_passes (Optional[int]) – As in
cake_dsa_sparse_attention_det(): accepted for parity, validated, without effect.backend (str) – Only
"cake".dkv_acc (Optional[torch.Tensor | Callable]) – Caller-owned FP32
[S_dst, >= 512]buffer (row stride a multiple of 4 elements and at least 512, 16-byte-aligned base) into which the backward addsdK + dVin place (columns0:512, one FP32 addition per element; the caller zeroes it). When given, thekvgradient of the autograd call isNone. A zero-argument callable that returns it – or an(accumulator, dkv_dst_map)pair – may be given instead, as incake_dsa_sparse_attention_det().dkv_dst_map (Optional[torch.Tensor]) – int32
[S]destination row ofdkv_accfor every key row (default identity). Repeated destinations are allowed and combine exactly. Every value must lie in[0, S_dst); the kernels do not range-check the map (seecheck_dst_map).dkv_direct (Optional[bool]) – As in
cake_dsa_sparse_attention_det(): accepted for parity, validated, without effect.check_dst_map (Optional[bool]) – As in
cake_dsa_sparse_attention_det().
- Returns:
out (torch.Tensor) – BF16
[T, 64, 512]; zero for fully masked rows.lse (torch.Tensor) – Only with
return_lse=True.
Notes
Gradients flow to
qandkv.dkvis formed in FP32 from the integer sums and handed over in FP32: throughdkv_accwhen given, otherwise as thekvgradient – which the autograd engine casts to the dtype of thekvleaf (BF16 for a BF16 leaf); callflashinfer.experimental.cake.cake_dsa_sparse_attention_det_train.cake_backend.backward_nopeor passdkv_accto keep the FP32 values. The backward reuses the storage of the savedo_loas scratch (seecake_dsa_sparse_attention_det()).The incoming
doutmay be a head-major view (strides[512, T * 512, 1], the layout an output projection hands back) or any strided[T, 64, 512]view under the layout rules ofq: it is read in place with no host copy.Host side as in
cake_dsa_sparse_attention_det()(the sameBINDING_CACHE, under keys disjoint from the rope entries).