flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet

flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet(x: Tensor, w: Tensor, eps: float = 1e-06, residual: Tensor | None = None)

Autograd-enabled fused BF16 RMSNorm (optionally with a fused residual add).

Parameters:
  • x (torch.Tensor) – BF16 [T, H] input on a supported CUDA device, or [..., H] under the layout rule of cake_rmsnorm_nondet_train_forward().

  • w (torch.Tensor) – BF16 [H] RMSNorm weight (FP32 on the routes that ship the FP32-weight programs); the weight gradient is returned in w.dtype.

  • eps (float) – Variance epsilon evaluated in FP32 by the kernel.

  • residual (torch.Tensor, optional) – BF16 update of x’s shape added to x before normalization. When provided, the normalized input h_new is returned with the output.

Returns:

BF16 y of x’s shape, or (y, h_new) when residual is given. Backward launches the kernels of cake_rmsnorm_nondet_train_backward().

Return type:

torch.Tensor or tuple[torch.Tensor, torch.Tensor]

Notes

This package’s kernels, whose weight gradient is not guaranteed bitwise reproducible (some token-count windows sum it over the CTAs in a scheduling-dependent order); the deterministic package’s autograd entry point is flashinfer.cake_rmsnorm_det_train.cake_rmsnorm_det(). The forward does not consult torch.use_deterministic_algorithms; while it is True the backward raises RuntimeError naming that entry point when it runs, or, with warn_only=True, warns once and runs this package’s backward.