flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_logprob

flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_logprob(X: Tensor, W: Tensor, labels: Tensor, *, chunk_size: int = 4096, backend: str = 'cake', compact_rows: bool | None = None, fuse_dw_cast: bool | None = None, dx_finalize: bool | None = None, dw_stream: bool | str | None = None, hidden_count: bool | None = None, gemm_tuning: Dict[str, Dict[str, Any]] | None = None, memory_mode: str | None = None, num_valid_rows: int | None = None, check_num_valid_rows: bool = False, temperature: float | Tensor | None = None, check_temperature: bool = False) → Tensor

Differentiable chunked selected-token log-probability with a slab-summed dX reduction, SM100 / SM103 / SM107 only.

See flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_logprob() for the computation and the arguments (memory_mode="keep_logits", temperature and check_temperature included), except as the module docstring notes (the dX reduction and the zero-gradient row skip). The backward sums the K slices of a sliced dX GEMM in slice order and stores the sum into the FP32 dX accumulator (logp keeps its fixed order); torch’s deterministic mode raises PyTorch’s alert before that reduction, as for cake_lm_head_nondet_train_loss(). With a temperature the backend runs the same temperature forms (logp the deterministic entry’s bits, dW too when no valid row has a zero dlogp, except a folded short tail and a call of the vocabulary-major backward, by default 2 to 4 chunks: another dW grouping within the accuracy budgets; dX the same products, a sliced chunk’s summed in slice order). The zero-gradient row skip runs the dX / dW GEMMs of the rows with a nonzero dlogp only, with or without a temperature. None and a Python number whose FP32 value is 1.0 run the call without a temperature; a bool is a ValueError. With memory_mode="keep_logits" a hidden_count left at None is passed down as None (the backend’s kept-logits form decides, see its set_kept_logits_form) unless FLASHINFER_CAKE_LM_HEAD_LOSS_HIDDEN_COUNT is "0" (False).