flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_finish¶
- flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_finish(X: Tensor, dY: Tensor, dU: Tensor, B: Tensor | None, fn: Tensor, scale: Tensor, base: Tensor, stats: Tensor, grad_stats: Tensor, *, rms_eps: float = 1e-05, hc_eps: float = 1e-06, sinkhorn_iters: int = 20, accumulate: bool = False, workspace: Tensor | None = None, dX: Tensor | None = None, dfn: Tensor | None = None, dscale: Tensor | None = None, dbase: Tensor | None = None) tuple[Tensor, Tensor, Tensor, Tensor]¶
Input gradient and parameter gradients:
(dX, dfn, dscale, dbase).The arguments and outputs are those of
cake_mhc_det_train_backward_finish(); the call runs the backend package’s backward, with no final reduction kernel. Every output repeats bit for bit from run to run on one device.dX,dscaleanddbaseare bitwise the deterministic values. Each element ofdfnfolds itsP <= 32per-chunk FP32 partialsx_p(bitwise the deterministic package’s) in ascending chunk order as a compensated FP32 sum, the caller’sdfnlast underaccumulate=True, with one rounding: within one FP32 rounding of the exact sum plusO(P^2 u^2) (|start| + sum |x_p|)(u = 2^-24). The deterministic package sums the same partials in FP64 and rounds once, sodfnis the deterministicdfnexcept where the exact sum lies within that last term of an FP32 rounding boundary (see the module docstring).- Parameters:
X (torch.Tensor) – BF16
[T, 4, H]contiguous residual streams of the forward (H = X.shape[2]).dY (torch.Tensor) – BF16
[T, 4, H]contiguous gradient ofY.dU (torch.Tensor) – BF16
[T, H]contiguous gradient ofUfrom the branch backward.B (torch.Tensor, optional) – BF16
[T, H]branch output of the forward; validated when given, not read.fn (torch.Tensor) – The parameters of the forward (FP32
[24, 4H],[3],[24]).scale (torch.Tensor) – The parameters of the forward (FP32
[24, 4H],[3],[24]).base (torch.Tensor) – The parameters of the forward (FP32
[24, 4H],[3],[24]).stats (torch.Tensor) – FP32
[T, 64]record of the forward (saved, or recomputed withcake_mhc_nondet_train_forward_stats()).grad_stats (torch.Tensor) – FP32
[T, 32]record ofcake_mhc_nondet_train_backward_prepare().rms_eps (float, float, int) – As in
cake_mhc_nondet_train_forward_prepare().hc_eps (float, float, int) – As in
cake_mhc_nondet_train_forward_prepare().sinkhorn_iters (float, float, int) – As in
cake_mhc_nondet_train_forward_prepare().accumulate (bool) –
False(default):dfn,dscaleanddbaseare written.True: the caller passes all three and the sums are added to their current values.workspace (torch.Tensor, optional) – Caller-owned
uint8buffer fromcake_mhc_nondet_train_backward_workspace(); a per-device default when omitted.dX (torch.Tensor, optional) – Preallocated BF16
[T, 4, H]contiguous output.dfn (torch.Tensor, optional) – Preallocated FP32
[24, 4H],[3],[24]contiguous outputs (required withaccumulate=True).dscale (torch.Tensor, optional) – Preallocated FP32
[24, 4H],[3],[24]contiguous outputs (required withaccumulate=True).dbase (torch.Tensor, optional) – Preallocated FP32
[24, 4H],[3],[24]contiguous outputs (required withaccumulate=True).
- Returns:
(dX, dfn, dscale, dbase): the BF16 input gradient and the FP32 parameter gradients.- Return type:
tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]