flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_prepare

flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_prepare(X: Tensor, fn: Tensor, scale: Tensor, base: Tensor, *, rms_eps: float = 1e-05, hc_eps: float = 1e-06, sinkhorn_iters: int = 20, out: Tensor | None = None, stats_out: Tensor | None = None, workspace: Tensor | None = None) → tuple[Tensor, Tensor]

Forward mapping, coefficients and collapse: (U, stats).

The arguments and outputs are those of cake_mhc_det_train_forward_prepare(). The call runs the backend package, whose forward runs copies of the deterministic programs, so U and stats are the deterministic package’s bits.

Parameters:
  • X (torch.Tensor) – BF16 [T, 4, H] contiguous residual streams; H = X.shape[2] is 4096, 5120 or 7168 on SM100 / SM103 and 4096 on SM107.

  • fn (torch.Tensor) – FP32 [24, 4H] mapping matrix, [3] group scales and [24] logit offsets.

  • scale (torch.Tensor) – FP32 [24, 4H] mapping matrix, [3] group scales and [24] logit offsets.

  • base (torch.Tensor) – FP32 [24, 4H] mapping matrix, [3] group scales and [24] logit offsets.

  • rms_eps (float, float, int) – Normalization epsilon (default 1e-5), coefficient epsilon (default 1e-6) and Sinkhorn iteration count (1 to 64, default 20).

  • hc_eps (float, float, int) – Normalization epsilon (default 1e-5), coefficient epsilon (default 1e-6) and Sinkhorn iteration count (1 to 64, default 20).

  • sinkhorn_iters (float, float, int) – Normalization epsilon (default 1e-5), coefficient epsilon (default 1e-6) and Sinkhorn iteration count (1 to 64, default 20).

  • out (torch.Tensor, optional) – Preallocated BF16 [T, H] contiguous U.

  • stats_out (torch.Tensor, optional) – Preallocated FP32 [T, 64] contiguous statistics record.

  • workspace (torch.Tensor, optional) – Caller-owned buffer from cake_mhc_nondet_train_forward_workspace() (hidden=X.shape[2]); a per-device default when omitted.

Returns:

(U, stats): BF16 U of shape [T, H] and the FP32 [T, 64] per-token record.

Return type:

tuple[torch.Tensor, torch.Tensor]