flashinfer.cake_mhc_head_nondet_train.cake_mhc_head_nondet_train_forward¶
- flashinfer.cake_mhc_head_nondet_train.cake_mhc_head_nondet_train_forward(X: Tensor, *, out: Tensor | None = None) Tensor¶
The model’s head: the mean of the four residual streams,
Y = mean_i X[:, i, :].Y[t, d] = BF16(((X[t, 0, d] + X[t, 1, d]) + X[t, 2, d] + X[t, 3, d]) * 0.25)with the sum in FP32 from zero in stream order and one rounding: bit for bit torch’sX.mean(dim=1)of a BF16X, signed zeros and the NaN encoding included.- Parameters:
X (torch.Tensor) – BF16
[T, 4, H]contiguous residual streams,Hin 4096, 5120 or 7168 (is_cake_mhc_head_nondet_train_supported()), 32-byte aligned.Tis a runtime scalar;T == 0returns an emptyYwithout launching.out (torch.Tensor, optional) – Preallocated BF16
[T, H]contiguousY(defaultNone: a new tensor); it must not overlapX.
- Returns:
BF16
Yof shape[T, H].The launch runs on the current torch stream of
X’s device and allocates nothing beyond the defaultout, so it can be captured in a CUDA graph once its program is loaded (one eager call first).- Return type:
torch.Tensor