flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_kl¶
- flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_kl(X: Tensor, W: Tensor, support_ids: Tensor, support_offsets: Tensor, teacher_logprobs: Tensor, *, k_cap: int, temperature: None | float | Tensor = 1.0, return_logprobs: bool = False, chunk_size: int | None = None, check_inputs: bool = False, backend: str = 'cake')¶
Unreduced forward KL from a detached teacher over a ragged candidate support, SM100 / SM103 / SM107 only.
KL_t = sum_j Q_j (lq_j - logp_j)over the candidates of rowtwithQ_j = exp(lq_j) > 0, for the FP32 teacher log-probabilitiesteacher_logprobs [M]aligned withsupport_ids(requires_grad=False;-infentries drop out) and the studentlogpofcake_lm_head_det_train_support_logprob()on the same support;0on a row of one candidate or none. Returns the FP32kl [T], or(kl, logp)with the differentiable packed studentlogp [M]whenreturn_logprobs=True. Weighting, reduction and anytau ** 2factor belong to the caller. Other arguments as incake_lm_head_det_train_support_logprob().- Parameters:
X (torch.Tensor) – Row-major BF16
[T, H]hidden states, withHa multiple of 256.W (torch.Tensor) – Contiguous BF16
[V, H]output weight, withVa multiple of 256.support_ids (torch.Tensor) – Packed int32 or int64 candidate ids
[M], unique within each row.support_offsets (torch.Tensor) – int32 or int64
[T + 1]row offsets, first 0 and lastM.teacher_logprobs (torch.Tensor) – Detached FP32
[M]teacher log-probabilities aligned withsupport_ids.return_logprobs (bool, default=False) – Also return the differentiable packed student log-probabilities.
k_cap (int) – Maximum candidates per row; rows exceeding it are rejected, not trimmed.
temperature (float or torch.Tensor, optional) – Positive finite scalar or FP32
[T]per-row temperature; default 1.0.chunk_size (int, optional) – Token chunk size for the dense pass and large-support weight gradient; default 4096.
check_inputs (bool, default=False) – Synchronize validation and raise
ValueErroron invalid input; otherwise use a device assertion.backend (str, default="cake") – Only
"cake"is supported.