flashinfer.moe_ep: Expert-Parallel Communication

A MoEEpCommunication dispatches each token to the ranks that own its experts and combines the expert outputs back on the token’s source rank. create_communication() builds one from a BootstrapConfig, which describes the expert-parallel group, and a backend config; the backends below run over NVLink (MNNVL) symmetric memory.

BootstrapConfig(world_size, rank[, stream, ...])	Inputs each backend needs to construct a Fleet.
MoEEpCommunication(bootstrap, params)	Self-contained MoE expert-parallel dispatch/combine backend.
MoEEpCommParams(num_experts, top_k, ...[, ...])	EP sizing shared by every communication backend.
MoEEpDispatchResult(hidden_states, topk_ids, ...)	Rank-major receive buffers produced by MoEEpCommunication.dispatch().
NVLinkOneSidedAlltoAll(bootstrap, params[, ...])	Rank-major dispatch/combine with NVLink one-sided put/get kernels.
NVLinkOneSidedConfig([backend_name, ...])	Options of NVLinkOneSidedAlltoAll.
CakeAlltoAll(bootstrap, params[, config])	Rank-major dispatch/combine with the generated Cake kernels.
CakeAlltoAllConfig([backend_name, ...])	Options of CakeAlltoAll.
NVLinkTwoSidedAlltoAll(bootstrap, params[, ...])	Rank-major dispatch/combine with NVLink two-sided all-to-all-v kernels.
NVLinkTwoSidedConfig([backend_name, ...])	Options of NVLinkTwoSidedAlltoAll.

create_communication(bootstrap, params, ...)

Instantiate the registered communication backend named by backend.