flashinfer.moe_ep.BootstrapConfig

class flashinfer.moe_ep.BootstrapConfig(world_size: int, rank: int, stream: int = 0, auto_bootstrap: bool = True, nccl_comm: int | None = None, process_group: 'torch.distributed.ProcessGroup' | None = None, tcp_store: 'torch.distributed.TCPStore' | None = None, *, device: int | None = None)

Bases: object

Inputs each backend needs to construct a Fleet.

Backends consume only the fields they care about (NCCL-EP uses nccl_comm / process_group + stream; NIXL-EP uses tcp_store). Carrying all of them here lets a single bootstrap config drive either backend.

world_size and rank describe the EP comm domain. When the host framework uses a non-WORLD process group (e.g. vLLM expert parallel within a PP stage), pass that group via process_group; mega kernels then use it for symmetric-memory setup and collectives instead of WORLD.

Communicator resolution (NCCL-EP), in priority order:

  • nccl_comm — an existing ncclComm_t (as an int). The Fleet adopts it (wraps without taking ownership; it is never destroyed or aborted by the Fleet), so a host — e.g. vLLM — can share the exact communicator its process group already owns.

  • process_group — a torch ProcessGroup to mirror. The Fleet creates a fresh NCCL communicator over that group’s membership (the torch-version-robust pattern), letting the caller target a specific EP subgroup rather than the default WORLD group.

  • neither set — mirror the default process group.