flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_forward

flashinfer.cake_moe_grouped_gemm_det_train.cake_moe_grouped_gemm_det_train_forward(x: Tensor, w: Tensor, offs: Tensor, out: Tensor | None = None) → Tensor

Y[offs[e-1]:offs[e]] = X[offs[e-1]:offs[e]] @ W[e].T for every group e.

The slice lower bound is 0 for e == 0 and offs[e-1] for e > 0. For m_indptr input, offs in these formulas means m_indptr[1:].

Parameters:
  • x (torch.Tensor) – [sum_m, K] bfloat16 rows of all groups, group after group.

  • w (torch.Tensor) – [E, N, K] bfloat16 weights, one [N, K] matrix per group.

  • offs (torch.Tensor) – [E] int32 cumulative end offsets on the device (offs[-1] == sum_m), or m_indptr [E + 1] with a leading 0; read by the kernel, never on the host.

  • out (Optional[torch.Tensor]) – [sum_m, N] bfloat16 output (allocated when omitted); may be row-padded.

Returns:

out. N % 256 == 0 and K % 64 == 0 are required.

Return type:

torch.Tensor

Notes

Prepares and launches in one call. For repeated launches over the same tensors (or CUDA Graph capture) use cake_moe_grouped_gemm_det_train_prepare_forward once and call launch() on the returned object.