flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_kl

flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_kl(X: Tensor, W: Tensor, support_ids: Tensor, support_offsets: Tensor, teacher_logprobs: Tensor, *, k_cap: int, temperature: None | float | Tensor = 1.0, return_logprobs: bool = False, chunk_size: int | None = None, check_inputs: bool = False, backend: str = 'cake')

Unreduced forward KL from a detached teacher over a ragged candidate support, SM100 / SM103 / SM107 only.

KL_t = sum_j Q_j (lq_j - logp_j) over the candidates of row t with Q_j = exp(lq_j) > 0, for the FP32 teacher log-probabilities teacher_logprobs [M] aligned with support_ids (requires_grad=False; -inf entries drop out) and the student logp of cake_lm_head_det_train_support_logprob() on the same support; 0 on a row of one candidate or none. Returns the FP32 kl [T], or (kl, logp) with the differentiable packed student logp [M] when return_logprobs=True. Weighting, reduction and any tau ** 2 factor belong to the caller. Other arguments as in cake_lm_head_det_train_support_logprob().

Parameters:
  • X (torch.Tensor) – Row-major BF16 [T, H] hidden states, with H a multiple of 256.

  • W (torch.Tensor) – Contiguous BF16 [V, H] output weight, with V a multiple of 256.

  • support_ids (torch.Tensor) – Packed int32 or int64 candidate ids [M], unique within each row.

  • support_offsets (torch.Tensor) – int32 or int64 [T + 1] row offsets, first 0 and last M.

  • teacher_logprobs (torch.Tensor) – Detached FP32 [M] teacher log-probabilities aligned with support_ids.

  • return_logprobs (bool, default=False) – Also return the differentiable packed student log-probabilities.

  • k_cap (int) – Maximum candidates per row; rows exceeding it are rejected, not trimmed.

  • temperature (float or torch.Tensor, optional) – Positive finite scalar or FP32 [T] per-row temperature; default 1.0.

  • chunk_size (int, optional) – Token chunk size for the dense pass and large-support weight gradient; default 4096.

  • check_inputs (bool, default=False) – Synchronize validation and raise ValueError on invalid input; otherwise use a device assertion.

  • backend (str, default="cake") – Only "cake" is supported.