flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_prepare

flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_prepare(dY: Tensor, B: Tensor, X: Tensor, stats: Tensor, *, dB: Tensor | None = None, grad_stats_out: Tensor | None = None) → tuple[Tensor, Tensor]

Branch-output gradient and the per-token gradient statistics: (dB, grad_stats).

The arguments and outputs are those of cake_mhc_det_train_backward_prepare(); the call runs the backend package (the deterministic package’s bits: this pass has no cross-token reduction).

Parameters:
  • dY (torch.Tensor) – BF16 [T, 4, H] contiguous gradient of Y.

  • B (torch.Tensor) – BF16 [T, H] contiguous branch output of the forward.

  • X (torch.Tensor) – BF16 [T, 4, H] contiguous residual streams of the forward.

  • stats (torch.Tensor) – FP32 [T, 64] record of the forward.

  • dB (torch.Tensor, optional) – Preallocated BF16 [T, H] contiguous output.

  • grad_stats_out (torch.Tensor, optional) – Preallocated FP32 [T, 32] contiguous output.

Returns:

(dB, grad_stats): BF16 dB of shape [T, H] and the FP32 [T, 32] record that cake_mhc_nondet_train_backward_finish() consumes.

Return type:

tuple[torch.Tensor, torch.Tensor]