flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_nope

flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det_nope(q: Tensor, kv: Tensor, indices: Tensor, *, topk_length: Tensor | None = None, softmax_scale: float | None = None, return_lse: bool = False, key_passes: int | None = None, backend: str = 'cake', dkv_acc: Tensor | Callable | None = None, dkv_dst_map: Tensor | None = None, dkv_direct: bool | None = None, check_dst_map: bool | None = None)

Differentiable top-k sparse MLA attention without a rope part (NoPE: the whole score is the 512-dimensional latent product) for training, 64 query heads, SM100 / SM103 / SM107 only; every output repeats bit for bit.

O = softmax(softmax_scale * (q . kv^T)) . kv over the keys a row’s indices select (K = V = the shared latent kv). The native kernels of this form read no rope operands and produce one combined gradient per key row, dkv = dK + dV, natural layout, FP32.

Parameters:
  • q (torch.Tensor) – BF16 [T, 64, 512] absorbed query latents; contiguous or a strided token / head view (unit innermost stride, head and token strides positive multiples of 8 elements, head stride at least 512, 16-byte-aligned base), consumed in place.

  • kv (torch.Tensor) – BF16 [S, 512] shared key/value latent, or its one-head form [S, 1, 512] (squeezed to [S, 512] on the host; the gradient reaches the [S, 1, 512] leaf through that view).

  • indices (torch.Tensor) – int32 [T, K_slots] global key rows into kv. Any K_slots >= 1; -1 or a value >= S marks an invalid slot, anywhere in the row. A row whose every slot is invalid gives out = 0, lse = -inf and zero gradients.

  • topk_length (Optional[torch.Tensor]) – int32 [T]; slots >= topk_length[t] are invalid regardless of their content. Without it the route follows the table size (cake_backend.NOPE_CONVERT_SLOT_LIMIT slots): below the limit the per-row lengths (last valid slot + 1) are derived by the forward kernel while it runs and saved for the backward with the rows as given; at or above it the rows are compacted on device first (valid keys first in slot order, then -1, the valid count as the length) and the compacted pair is saved. Hole-free rows give the same bytes on either route.

  • softmax_scale (Optional[float]) – Defaults to 1 / 16.

  • return_lse (bool) – Also return the natural-log logsumexp [T, 64] FP32 of the scaled scores over the valid keys (-inf for fully masked rows).

  • key_passes (Optional[int]) – As in cake_dsa_sparse_attention_det(): accepted for parity, validated, without effect.

  • backend (str) – Only "cake".

  • dkv_acc (Optional[torch.Tensor | Callable]) – Caller-owned FP32 [S_dst, >= 512] buffer (row stride a multiple of 4 elements and at least 512, 16-byte-aligned base) into which the backward adds dK + dV in place (columns 0:512, one FP32 addition per element; the caller zeroes it). When given, the kv gradient of the autograd call is None. A zero-argument callable that returns it – or an (accumulator, dkv_dst_map) pair – may be given instead, as in cake_dsa_sparse_attention_det().

  • dkv_dst_map (Optional[torch.Tensor]) – int32 [S] destination row of dkv_acc for every key row (default identity). Repeated destinations are allowed and combine exactly. Every value must lie in [0, S_dst); the kernels do not range-check the map (see check_dst_map).

  • dkv_direct (Optional[bool]) – As in cake_dsa_sparse_attention_det(): accepted for parity, validated, without effect.

  • check_dst_map (Optional[bool]) – As in cake_dsa_sparse_attention_det().

Returns:

  • out (torch.Tensor) – BF16 [T, 64, 512]; zero for fully masked rows.

  • lse (torch.Tensor) – Only with return_lse=True.

Notes

Gradients flow to q and kv. dkv is formed in FP32 from the integer sums and handed over in FP32: through dkv_acc when given, otherwise as the kv gradient – which the autograd engine casts to the dtype of the kv leaf (BF16 for a BF16 leaf); call flashinfer.experimental.cake.cake_dsa_sparse_attention_det_train.cake_backend.backward_nope or pass dkv_acc to keep the FP32 values. The backward reuses the storage of the saved o_lo as scratch (see cake_dsa_sparse_attention_det()).

The incoming dout may be a head-major view (strides [512, T * 512, 1], the layout an output projection hands back) or any strided [T, 64, 512] view under the layout rules of q: it is read in place with no host copy.

Host side as in cake_dsa_sparse_attention_det() (the same BINDING_CACHE, under keys disjoint from the rope entries).