flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_support_kl¶
- flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_support_kl(X: Tensor, W: Tensor, support_ids: Tensor, support_offsets: Tensor, teacher_logprobs: Tensor, *, k_cap: int, temperature: None | float | Tensor = 1.0, return_logprobs: bool = False, chunk_size: int | None = None, check_inputs: bool = False, backend: str = 'cake')¶
Unreduced forward KL from a detached teacher over a ragged candidate support, SM100 / SM103 / SM107 only.
See
flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_kl()for the computation and the arguments; the gradients are as incake_lm_head_nondet_train_support_logprob().