flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_forward¶
- flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_forward(x: Tensor, w: Tensor, offs: Tensor, out: Tensor | None = None) Tensor¶
Y[offs[e-1]:offs[e]] = X[offs[e-1]:offs[e]] @ W[e].Tfor every groupe.The slice lower bound is
0fore == 0andoffs[e-1]fore > 0. Form_indptrinput,offsin these formulas meansm_indptr[1:].- Parameters:
x (torch.Tensor) –
[sum_m, K]bfloat16 rows of all groups, group after group.w (torch.Tensor) –
[E, N, K]bfloat16 weights, one[N, K]matrix per group.offs (torch.Tensor) –
[E]int32 cumulative end offsets on the device (offs[-1] == sum_m), orm_indptr[E + 1]with a leading 0; read by the kernel, never on the host.out (Optional[torch.Tensor]) –
[sum_m, N]bfloat16 output (allocated when omitted); may be row-padded.
- Returns:
out.N % 256 == 0andK % 64 == 0are required.- Return type:
torch.Tensor
Notes
Prepares and launches in one call. For repeated launches over the same tensors (or CUDA Graph capture) use
cake_moe_grouped_gemm_det_train_prepare_forwardonce and calllaunch()on the returned object.