flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_logprob¶
- flashinfer.cake_lm_head_nondet_train.cake_lm_head_nondet_train_logprob(X: Tensor, W: Tensor, labels: Tensor, *, chunk_size: int = 4096, backend: str = 'cake', compact_rows: bool | None = None, fuse_dw_cast: bool | None = None, dx_finalize: bool | None = None, dw_stream: bool | str | None = None, hidden_count: bool | None = None, gemm_tuning: Dict[str, Dict[str, Any]] | None = None, memory_mode: str | None = None, num_valid_rows: int | None = None, check_num_valid_rows: bool = False, temperature: float | Tensor | None = None, check_temperature: bool = False) Tensor¶
Differentiable chunked selected-token log-probability with a slab-summed dX reduction, SM100 / SM103 / SM107 only.
See
flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_logprob()for the computation and the arguments (memory_mode="keep_logits",temperatureandcheck_temperatureincluded), except as the module docstring notes (the dX reduction and the zero-gradient row skip). The backward sums the K slices of a sliced dX GEMM in slice order and stores the sum into the FP32dXaccumulator (logpkeeps its fixed order); torch’s deterministic mode raises PyTorch’s alert before that reduction, as forcake_lm_head_nondet_train_loss(). With a temperature the backend runs the same temperature forms (logpthe deterministic entry’s bits,dWtoo when no valid row has a zerodlogp, except a folded short tail and a call of the vocabulary-major backward, by default 2 to 4 chunks: anotherdWgrouping within the accuracy budgets;dXthe same products, a sliced chunk’s summed in slice order). The zero-gradient row skip runs the dX / dW GEMMs of the rows with a nonzerodlogponly, with or without a temperature.Noneand a Python number whose FP32 value is 1.0 run the call without a temperature; aboolis aValueError. Withmemory_mode="keep_logits"ahidden_countleft atNoneis passed down asNone(the backend’s kept-logits form decides, see itsset_kept_logits_form) unlessFLASHINFER_CAKE_LM_HEAD_LOSS_HIDDEN_COUNTis"0"(False).