flashinfer.comm.free_shared_buffer

flashinfer.comm.free_shared_buffer(pointers: List[int], group: ProcessGroup | None = None) → None

Free a shared buffer previously created by create_shared_buffer().

Collective: every rank in the group must call this together. Callers must ensure no kernel is still using the buffers (synchronize the streams that touched them) before calling.

Teardown order matters for CUDA IPC: every rank first closes the peer mappings it opened with cudaIpcOpenMemHandle, a barrier confirms all mappings are closed everywhere, and only then does each rank cudaFree its own allocation — freeing memory still IPC-mapped in a peer process is undefined behavior.

Parameters:

  • pointers (list[int]) – Per-rank pointer list returned by create_shared_buffer().

  • group (torch.distributed.ProcessGroup, optional) – Process group used during allocation. Defaults to dist.group.WORLD.