flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_finish

flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_finish(X: Tensor, dY: Tensor, dU: Tensor, B: Tensor | None, fn: Tensor, scale: Tensor, base: Tensor, stats: Tensor, grad_stats: Tensor, *, rms_eps: float = 1e-05, hc_eps: float = 1e-06, sinkhorn_iters: int = 20, accumulate: bool = False, workspace: Tensor | None = None, dX: Tensor | None = None, dfn: Tensor | None = None, dscale: Tensor | None = None, dbase: Tensor | None = None) → tuple[Tensor, Tensor, Tensor, Tensor]

Input gradient and parameter gradients: (dX, dfn, dscale, dbase).

The arguments and outputs are those of cake_mhc_det_train_backward_finish(); the call runs the backend package’s backward, with no final reduction kernel. Every output repeats bit for bit from run to run on one device. dX, dscale and dbase are bitwise the deterministic values. Each element of dfn folds its P <= 32 per-chunk FP32 partials x_p (bitwise the deterministic package’s) in ascending chunk order as a compensated FP32 sum, the caller’s dfn last under accumulate=True, with one rounding: within one FP32 rounding of the exact sum plus O(P^2 u^2) (|start| + sum |x_p|) (u = 2^-24). The deterministic package sums the same partials in FP64 and rounds once, so dfn is the deterministic dfn except where the exact sum lies within that last term of an FP32 rounding boundary (see the module docstring).

Parameters:
  • X (torch.Tensor) – BF16 [T, 4, H] contiguous residual streams of the forward (H = X.shape[2]).

  • dY (torch.Tensor) – BF16 [T, 4, H] contiguous gradient of Y.

  • dU (torch.Tensor) – BF16 [T, H] contiguous gradient of U from the branch backward.

  • B (torch.Tensor, optional) – BF16 [T, H] branch output of the forward; validated when given, not read.

  • fn (torch.Tensor) – The parameters of the forward (FP32 [24, 4H], [3], [24]).

  • scale (torch.Tensor) – The parameters of the forward (FP32 [24, 4H], [3], [24]).

  • base (torch.Tensor) – The parameters of the forward (FP32 [24, 4H], [3], [24]).

  • stats (torch.Tensor) – FP32 [T, 64] record of the forward (saved, or recomputed with cake_mhc_nondet_train_forward_stats()).

  • grad_stats (torch.Tensor) – FP32 [T, 32] record of cake_mhc_nondet_train_backward_prepare().

  • rms_eps (float, float, int) – As in cake_mhc_nondet_train_forward_prepare().

  • hc_eps (float, float, int) – As in cake_mhc_nondet_train_forward_prepare().

  • sinkhorn_iters (float, float, int) – As in cake_mhc_nondet_train_forward_prepare().

  • accumulate (bool) – False (default): dfn, dscale and dbase are written. True: the caller passes all three and the sums are added to their current values.

  • workspace (torch.Tensor, optional) – Caller-owned uint8 buffer from cake_mhc_nondet_train_backward_workspace(); a per-device default when omitted.

  • dX (torch.Tensor, optional) – Preallocated BF16 [T, 4, H] contiguous output.

  • dfn (torch.Tensor, optional) – Preallocated FP32 [24, 4H], [3], [24] contiguous outputs (required with accumulate=True).

  • dscale (torch.Tensor, optional) – Preallocated FP32 [24, 4H], [3], [24] contiguous outputs (required with accumulate=True).

  • dbase (torch.Tensor, optional) – Preallocated FP32 [24, 4H], [3], [24] contiguous outputs (required with accumulate=True).

Returns:

(dX, dfn, dscale, dbase): the BF16 input gradient and the FP32 parameter gradients.

Return type:

tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]