flashinfer.cake_moe_grouped_gemm_nondet_train.cake_moe_grouped_gemm_nondet_train_prepare_wgrad

flashinfer.cake_moe_grouped_gemm_nondet_train.cake_moe_grouped_gemm_nondet_train_prepare_wgrad(g: torch.Tensor, x: torch.Tensor, offs: torch.Tensor, out: torch.Tensor | None = None, out_dtype: torch.dtype | None = None, num_groups: int | None = None, *, accumulate: bool = False) → CakeMoEGroupedGemmNondetLaunch

Prepare dW[E, N, K] = grouped G[e].T @ X[e] (bf16 in; bf16 or fp32 out) with this package’s programs.

g: [sum_m, N] bf16; x: [sum_m, K] bf16; offs: int32 [E] device end offsets (E = offs.numel() unless num_groups is given, in which case an m_indptr of E + 1 entries is accepted too); N % 256 == 0 and K a multiple of the selected k tile (256, or 512 when it is chosen). Empty groups produce exact zeros. out_dtype defaults to the dtype of out (bf16 when neither is given).

With the 512-wide k tile a split tail is finished in the same launch by the ordered one-launch tail (plan["ordered_tail"] and plan["tail_splits"] > 1: each unit stores its fp32 partial to a slot of tail_sums and the tile’s last unit sums the slots in slot order); with the 256-wide k tile the deterministic program and its ordered tail reduce run. Either way dW repeats bit for bit for the same inputs, shapes, device SM count and configuration, and no launch alerts under torch.use_deterministic_algorithms(True) (the alert serves only an architecture listed in unordered_tail_archs: module docstring).

accumulate=True adds instead: every launch computes out[e] <- out[e] + G[e].T @ X[e] into the caller’s out (required, bf16 or fp32; out_dtype None or out.dtype; rows and experts that do not overlap, no memory shared with g, x or offs; ValueError otherwise). Empty groups are not written by the fused route, and no row (sum_m == 0) launches nothing and leaves out unchanged. plan["accumulate"] is "fused" (one launch of the stage’s accumulate build, stages == (stage + ACC_SUFFIX,)) or "composed" (the overwrite into a scratch of out.dtype owned by the launch, then out.add_; plan["accumulate_reason"] says why; keep the launch alive while a graph that captured it may replay); every other plan key is the overwrite’s (module docstring, accumulate mode). Each launch() and each replay of a graph that captured it adds again.

Parameters:
  • g (torch.Tensor) – BF16 [sum_m, N] packed output-gradient rows.

  • x (torch.Tensor) – BF16 [sum_m, K] packed activation rows.

  • offs (torch.Tensor) – int32 device cumulative end offsets [E], or leading-zero m_indptr [E + 1].

  • out (torch.Tensor, optional) – Caller-owned [E, N, K] output; required when accumulating.

  • out_dtype (torch.dtype, optional) – torch.bfloat16 or torch.float32; defaults to out.dtype or bfloat16 without out.

  • num_groups (int, optional) – Expert count E; must be supplied for m_indptr input.

  • accumulate (bool, default=False) – Add the weight gradient into out instead of overwriting it.