flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_logprob

flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_logprob(X: Tensor, W: Tensor, support_ids: Tensor, support_offsets: Tensor, labels: Tensor | None = None, *, k_cap: int, temperature: None | float | Tensor = 1.0, chunk_size: int | None = None, check_inputs: bool = False, backend: str = 'cake') → Tensor

Differentiable log-probabilities over a ragged candidate support per token, SM100 / SM103 / SM107 only.

The candidates of row t are support_ids[support_offsets[t]:support_offsets[t + 1]] (int32 or int64 ids, unique inside a row, any order; int32 or int64 offsets, support_offsets[0] = 0, nondecreasing, support_offsets[T] = M; at most k_cap per row, never trimmed). With z = X @ W^T as a BF16 GEMM (FP32 accumulation, one rounding) and the temperature tau_t: logp_j = z_j / tau_t - logsumexp_{k in row t}(z_k / tau_t) in FP32, 0 on a row of one candidate or none. labels is None: the packed FP32 logp [M] in caller order; otherwise the selected FP32 logp [T] at the int64 labels (-100: 0 with zero gradient; any other label must be one of the row’s candidates). temperature: a finite positive number or an FP32 [T] tensor on X’s device (not differentiable; checked on every row). chunk_size: the token chunk of the dense pass form and of the large-support weight gradient (default 4096). Invalid support is detected on the device: check_inputs=False queues an asynchronous assertion (no synchronization; a violation ends the CUDA context), check_inputs=True reads the validation word back once and raises ValueError naming the rule and the row. Gradients flow to X and W (BF16, FP32 accumulation, one cast each); outputs and gradients are bitwise reproducible run to run. X [T, H] BF16 row-major (H a multiple of 256), W [V, H] BF16 contiguous (V a multiple of 256, at most 1572864).

Parameters:
  • X (torch.Tensor) – Row-major BF16 [T, H] hidden states, with H a multiple of 256.

  • W (torch.Tensor) – Contiguous BF16 [V, H] output weight, with V a multiple of 256.

  • support_ids (torch.Tensor) – Packed int32 or int64 candidate ids [M], unique within each row.

  • support_offsets (torch.Tensor) – int32 or int64 [T + 1] row offsets, first 0 and last M.

  • labels (torch.Tensor, optional) – int64 [T] selected candidate ids (-100 ignores a row); None returns all candidate log-probabilities.

  • k_cap (int) – Maximum candidates per row; rows exceeding it are rejected, not trimmed.

  • temperature (float or torch.Tensor, optional) – Positive finite scalar or FP32 [T] per-row temperature; default 1.0.

  • chunk_size (int, optional) – Token chunk size for the dense pass and large-support weight gradient; default 4096.

  • check_inputs (bool, default=False) – Synchronize validation and raise ValueError on invalid input; otherwise use a device assertion.

  • backend (str, default="cake") – Only "cake" is supported.