flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_logprob¶
- flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_support_logprob(X: Tensor, W: Tensor, support_ids: Tensor, support_offsets: Tensor, labels: Tensor | None = None, *, k_cap: int, temperature: None | float | Tensor = 1.0, chunk_size: int | None = None, check_inputs: bool = False, backend: str = 'cake') Tensor¶
Differentiable log-probabilities over a ragged candidate support per token, SM100 / SM103 / SM107 only.
The candidates of row
taresupport_ids[support_offsets[t]:support_offsets[t + 1]](int32 or int64 ids, unique inside a row, any order; int32 or int64 offsets,support_offsets[0] = 0, nondecreasing,support_offsets[T] = M; at mostk_capper row, never trimmed). Withz = X @ W^Tas a BF16 GEMM (FP32 accumulation, one rounding) and the temperaturetau_t:logp_j = z_j / tau_t - logsumexp_{k in row t}(z_k / tau_t)in FP32,0on a row of one candidate or none.labels is None: the packed FP32logp [M]in caller order; otherwise the selected FP32logp [T]at the int64labels(-100: 0 with zero gradient; any other label must be one of the row’s candidates).temperature: a finite positive number or an FP32[T]tensor onX’s device (not differentiable; checked on every row).chunk_size: the token chunk of the dense pass form and of the large-support weight gradient (default 4096). Invalid support is detected on the device:check_inputs=Falsequeues an asynchronous assertion (no synchronization; a violation ends the CUDA context),check_inputs=Truereads the validation word back once and raisesValueErrornaming the rule and the row. Gradients flow toXandW(BF16, FP32 accumulation, one cast each); outputs and gradients are bitwise reproducible run to run.X [T, H]BF16 row-major (Ha multiple of 256),W [V, H]BF16 contiguous (Va multiple of 256, at most 1572864).- Parameters:
X (torch.Tensor) – Row-major BF16
[T, H]hidden states, withHa multiple of 256.W (torch.Tensor) – Contiguous BF16
[V, H]output weight, withVa multiple of 256.support_ids (torch.Tensor) – Packed int32 or int64 candidate ids
[M], unique within each row.support_offsets (torch.Tensor) – int32 or int64
[T + 1]row offsets, first 0 and lastM.labels (torch.Tensor, optional) – int64
[T]selected candidate ids (-100ignores a row);Nonereturns all candidate log-probabilities.k_cap (int) – Maximum candidates per row; rows exceeding it are rejected, not trimmed.
temperature (float or torch.Tensor, optional) – Positive finite scalar or FP32
[T]per-row temperature; default 1.0.chunk_size (int, optional) – Token chunk size for the dense pass and large-support weight gradient; default 4096.
check_inputs (bool, default=False) – Synchronize validation and raise
ValueErroron invalid input; otherwise use a device assertion.backend (str, default="cake") – Only
"cake"is supported.