flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet¶
- flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet(x: Tensor, w: Tensor, eps: float = 1e-06, residual: Tensor | None = None)¶
Autograd-enabled fused BF16 RMSNorm (optionally with a fused residual add).
- Parameters:
x (torch.Tensor) – BF16
[T, H]input on a supported CUDA device, or[..., H]under the layout rule ofcake_rmsnorm_nondet_train_forward().w (torch.Tensor) – BF16
[H]RMSNorm weight (FP32 on the routes that ship the FP32-weight programs); the weight gradient is returned inw.dtype.eps (float) – Variance epsilon evaluated in FP32 by the kernel.
residual (torch.Tensor, optional) – BF16 update of
x’s shape added toxbefore normalization. When provided, the normalized inputh_newis returned with the output.
- Returns:
BF16
yofx’s shape, or(y, h_new)whenresidualis given. Backward launches the kernels ofcake_rmsnorm_nondet_train_backward().- Return type:
torch.Tensor or tuple[torch.Tensor, torch.Tensor]
Notes
This package’s kernels, whose weight gradient is not guaranteed bitwise reproducible (some token-count windows sum it over the CTAs in a scheduling-dependent order); the deterministic package’s autograd entry point is
flashinfer.cake_rmsnorm_det_train.cake_rmsnorm_det(). The forward does not consulttorch.use_deterministic_algorithms; while it isTruethe backward raisesRuntimeErrornaming that entry point when it runs, or, withwarn_only=True, warns once and runs this package’s backward.