flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_prepare_wgrad¶
- flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_prepare_wgrad(g: torch.Tensor, x: torch.Tensor, offs: torch.Tensor, out: torch.Tensor | None = None, out_dtype: torch.dtype | None = None, num_groups: int | None = None, *, accumulate: bool = False) CakeMoEGroupedGemmDetLaunch¶
Prepare
dW[E, N, K] = grouped G[e].T @ X[e](bf16 in; bf16 or fp32 out), deterministic.g:[sum_m, N]bf16;x:[sum_m, K]bf16;offs: int32[E]device end offsets (E = offs.numel()unlessnum_groupsis given, in which case anm_indptrofE + 1entries is accepted too);N % 256 == 0andKa multiple of the selected k tile (256, or 512 when it is chosen). Empty groups produce exact zeros.out_dtypedefaults to the dtype ofout(bf16 when neither is given).accumulate=True(keyword-only, defaultFalse) adds instead: every launch computesout[e] <- out[e] + G[e].T @ X[e]into the caller’sout(required, bf16 or fp32;out_dtypeNone orout.dtype; rows and experts that do not overlap, no memory shared withg,xoroffs;ValueErrorotherwise, before anything is allocated or bound), sooutholds AccumulateGrad’s bits of the deterministic weight gradient in its dtype, bitwise reproducible run to run (an empty group adds exact zeros: its values stay,-0.0becomes+0.0).plan["accumulate"]is"fused"(the accumulate builds of the main stage and, for a split tail, of the tail reduce add intooutin place:stagesis the overwrite’s withACC_SUFFIXon each) or"composed"(the overwrite of this plan into a contiguous scratch ofout.dtype,E * N * Kelements that this preparation allocates and the launch owns, thenout.add_(scratch)on the same stream;plan["accumulate_reason"]says why); both give the same bits, and every other plan key is the overwrite’s (module docstring, accumulate mode). No row (sum_m == 0) launches nothing and leavesoutunchanged. Eachlaunch()and each replay of a graph that captured it adds again; keep the launch alive while such a graph may replay.- Parameters:
g (torch.Tensor) – BF16
[sum_m, N]packed output-gradient rows.x (torch.Tensor) – BF16
[sum_m, K]packed activation rows.offs (torch.Tensor) – int32 device cumulative end offsets
[E], or leading-zerom_indptr [E + 1].out (torch.Tensor, optional) – Caller-owned
[E, N, K]output; required when accumulating.out_dtype (torch.dtype, optional) –
torch.bfloat16ortorch.float32; defaults toout.dtypeor bfloat16 withoutout.num_groups (int, optional) – Expert count
E; must be supplied form_indptrinput.accumulate (bool, default=False) – Add the weight gradient into
outinstead of overwriting it.