flashinfer.cake_lm_head_nondet_train

cake_lm_head_nondet_train holds the non-deterministic Cake training entries of the chunked LM-head projection with a cross-entropy or policy loss, its per-token log-probabilities, and the log-probabilities / KL over a ragged candidate support, on SM100 (B200 / GB200), SM103 (B300 / GB300) and SM107 (R200). The entries take the arguments of the deterministic module flashinfer.cake_lm_head_det_train and keep its precision boundaries. Their bits may differ from the deterministic module’s, within the accuracy budgets, where the default path forms an output in another order: a sliced dX GEMM stores its K slices into an FP32 slab and a slab sum adds them in slice order (dX); zero-weight policy rows and zero-dlogp log-probability rows skip the gradient GEMMs (loss / dW); a call of fewer than three chunks folds a short last chunk into the chunk before it (loss / dW); a one-row cross-entropy loss call at H = 6144 runs one-pass programs (loss / logp / lse / dW); a log-probability backward of 2 to 4 chunks runs vocabulary-major by the README’s rule (dX / dW); and the candidate-support dense dX uses a wider vocabulary block (dX). No entry takes a deterministic keyword. The module makes no run-to-run bitwise promise for the gradients, and keeps PyTorch’s alert as part of that contract: under torch.use_deterministic_algorithms(True) an entry raises the alert right before a K-sliced dX reduction and never falls back to the deterministic module. The backend package is flashinfer.experimental.cake.cake_lm_head_nondet_train; its README lists the output differences.

cake_lm_head_nondet_train_loss(X, W, labels, *)

Differentiable chunked LM-head projection + loss with a memory-bounded backward and a slab-summed dX reduction, SM100 / SM103 / SM107 only.

cake_lm_head_nondet_train_logprob(X, W, ...)

Differentiable chunked selected-token log-probability with a slab-summed dX reduction, SM100 / SM103 / SM107 only.

cake_lm_head_nondet_train_preload([device, ...])

Do part of the one-time work of a process's first chunked LM-head call ahead of it, then wait for the device once (SM100 / SM103 / SM107 only).

cake_lm_head_nondet_train_support_logprob(X, ...)

Differentiable log-probabilities over a ragged candidate support per token, SM100 / SM103 / SM107 only.

cake_lm_head_nondet_train_support_kl(X, W, ...)

Unreduced forward KL from a detached teacher over a ragged candidate support, SM100 / SM103 / SM107 only.