flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_prepare¶
- flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_forward_prepare(X: Tensor, fn: Tensor, scale: Tensor, base: Tensor, *, rms_eps: float = 1e-05, hc_eps: float = 1e-06, sinkhorn_iters: int = 20, out: Tensor | None = None, stats_out: Tensor | None = None, workspace: Tensor | None = None) tuple[Tensor, Tensor]¶
Forward mapping, coefficients and collapse:
(U, stats).The arguments and outputs are those of
cake_mhc_det_train_forward_prepare(). The call runs the backend package, whose forward runs copies of the deterministic programs, soUandstatsare the deterministic package’s bits.- Parameters:
X (torch.Tensor) – BF16
[T, 4, H]contiguous residual streams;H = X.shape[2]is 4096, 5120 or 7168 on SM100 / SM103 and 4096 on SM107.fn (torch.Tensor) – FP32
[24, 4H]mapping matrix,[3]group scales and[24]logit offsets.scale (torch.Tensor) – FP32
[24, 4H]mapping matrix,[3]group scales and[24]logit offsets.base (torch.Tensor) – FP32
[24, 4H]mapping matrix,[3]group scales and[24]logit offsets.rms_eps (float, float, int) – Normalization epsilon (default
1e-5), coefficient epsilon (default1e-6) and Sinkhorn iteration count (1 to 64, default20).hc_eps (float, float, int) – Normalization epsilon (default
1e-5), coefficient epsilon (default1e-6) and Sinkhorn iteration count (1 to 64, default20).sinkhorn_iters (float, float, int) – Normalization epsilon (default
1e-5), coefficient epsilon (default1e-6) and Sinkhorn iteration count (1 to 64, default20).out (torch.Tensor, optional) – Preallocated BF16
[T, H]contiguousU.stats_out (torch.Tensor, optional) – Preallocated FP32
[T, 64]contiguous statistics record.workspace (torch.Tensor, optional) – Caller-owned buffer from
cake_mhc_nondet_train_forward_workspace()(hidden=X.shape[2]); a per-device default when omitted.
- Returns:
(U, stats): BF16Uof shape[T, H]and the FP32[T, 64]per-token record.- Return type:
tuple[torch.Tensor, torch.Tensor]