flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det

flashinfer.cake_dsa_sparse_attention_det_train.cake_dsa_sparse_attention_det(q_latent: Tensor, q_rope: Tensor, kv_latent: Tensor, k_rope: Tensor, indices: Tensor, *, topk_length: Tensor | None = None, softmax_scale: float | None = None, return_lse: bool = False, key_passes: int | None = None, backend: str = 'cake', dkv_acc: Tensor | Callable | None = None, dkv_dst_map: Tensor | None = None, dkv_direct: bool | None = None, check_dst_map: bool | None = None)

Differentiable top-k sparse MLA attention with absorbed queries (DeepSeek Sparse Attention) for training, 64 query heads, SM100 / SM103 / SM107 only; every output repeats bit for bit.

O = softmax(softmax_scale * (q_rope . k_rope^T + q_latent . kv_latent^T)) . kv_latent over the keys a row’s indices select (K = V = the latent).

Strided inputs are consumed in place (no host copy): the kernels read the query operands through TMA descriptors encoded from the tensor’s own head and token strides and gather the key operands by row stride. Admitted layouts: unit innermost stride; head / token / key-row strides that are positive multiples of 8 elements (16 bytes, the TMA descriptor stride granule); head stride and key row stride at least the slice width; a 16-byte-aligned base address. Outputs equal the contiguous path bitwise.

Parameters:
  • q_latent (torch.Tensor) – BF16 [T, 64, 512] absorbed query latents; contiguous or a view of a packed [T, 64, 576] query (head stride 576).

  • q_rope (torch.Tensor) – BF16 [T, 64, 64] query rope part; contiguous, a view of the packed [T, 64, 576] query, or the 192:256 channel slice of the pre-absorption [T, 64, 256] query (head stride 256, token stride 16384, storage offset 192 elements).

  • kv_latent (torch.Tensor) – BF16 [S, 512] key/value latent (shared K = V); columns 0:512 of a packed [S, 576] or [S, 704] row (a 128-channel indexer key stored alongside) are accepted.

  • k_rope (torch.Tensor) – BF16 [S, 64] key rope part; columns 512:576 of the same packed row are accepted.

  • indices (torch.Tensor) – int32 [T, topk] global key rows into kv_latent / k_rope. -1 or a value >= S marks an invalid slot; invalid slots may appear anywhere in the row. Any positive topk.

  • topk_length (Optional[torch.Tensor]) – int32 [T]; slots >= topk_length[t] are invalid regardless of their content.

  • softmax_scale (Optional[float]) – Defaults to 576 ** -0.5.

  • return_lse (bool) – Also return the natural-log logsumexp [T, 64] FP32 of the scaled scores over the valid keys (-inf for fully masked rows).

  • key_passes (Optional[int]) – Validated as in the non-deterministic package (None or 1 <= key_passes <= S). None lets that package’s key-pass policy choose; a count above 1 runs the backward’s main stage as that many key-range passes (a compaction and a main pass each, with their scratch in the backward’s workspace). A scheduling choice only: the gradients are bit for bit those of one pass.

  • backend (str) – Only "cake".

  • dkv_acc (Optional[torch.Tensor | Callable]) – Caller-owned FP32 [S_dst, >= 576] buffer (row stride a multiple of 4 elements, 16-byte aligned) into which the backward adds the key gradients in place (the caller zeroes it): latent dimensions in columns 0:512, rope dimensions in 512:576, one FP32 addition per element (an element that receives no contribution keeps its bits). When given, the kv_latent / k_rope gradients of the autograd call are None. Instead of the tensor, a zero-argument callable that returns it – or an (accumulator, dkv_dst_map) pair – may be given; the backward calls it, so the accumulator need not exist at forward time (under activation checkpointing a tensor passed at forward time is alive for every layer at once). The autograd entries save every tensor they produce for the backward through ctx.save_for_backward and retain nothing else of a checkpointed forward; a tensor dkv_acc stays caller-owned.

  • dkv_dst_map (Optional[torch.Tensor]) – int32 [S] destination row of dkv_acc for every key row (default identity). Repeated destinations are allowed (the rows of a context-parallel window that map onto one parameter row) and combine exactly, as integers, before the one FP32 addition. Every value must lie in [0, S_dst); the kernels do not range-check the map (see check_dst_map).

  • dkv_direct (Optional[bool]) – Accepted for signature parity with the non-deterministic package (None, True or False) and without effect: the accumulation into dkv_acc has one form here.

  • check_dst_map (Optional[bool]) – Validate the values of dkv_dst_map (every destination row in [0, S_dst)) on every backward, at the cost of one device synchronization, raising ValueError on a violation (skipped during CUDA graph capture). None (default) reads FLASHINFER_CAKE_DSA_CHECK_DST_MAP: unset, empty or 0 (its default) means no check, anything else enables it. True / False decide explicitly and ignore the variable.

Returns:

  • out (torch.Tensor) – BF16 [T, 64, 512]; zero for fully masked rows.

  • lse (torch.Tensor) – Only with return_lse=True.

Notes

Gradients flow to q_latent, q_rope, kv_latent and k_rope. BF16 operands into the tensor cores, FP32 accumulation; the backward recomputes the scores from the BF16 inputs and forms the exact delta from the saved output residual o_lo. The dK / dV contributions are rounded once to the row’s integer grid and summed as integers; each gradient element is then formed with one more rounding (and, with dkv_acc, one FP32 addition). The backward reuses the storage of the saved o_lo as its integer accumulator when it fits (S_dst up to about 27.7 T rows) and allocates it otherwise; a second backward through the same graph (retain_graph=True) first recomputes the forward into the saved tensors. The kernels apply no positional mask: the index rows define the key set.

Host side: the inputs are validated and bound on the first call for an input geometry (shape, stride, dtype and alignment of every input plus the scale); later calls launch from the remembered plans with freshly allocated outputs (flashinfer.experimental.cake.cake_dsa_sparse_attention_det_train.cake_backend.BINDING_CACHE, 64 bindings by default, least recently used first out; FLASHINFER_CAKE_DSA_TRAIN_BINDING_CACHE_CAPACITY – a positive integer, read when this module is imported – sets the capacity through BINDING_CACHE.configure(capacity=...)). No host synchronization unless check_dst_map asks for one.