flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_partition¶
- flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_partition(cu_seqlens_global_cpu: Any, world_size: int, layout: str = 'contiguous', fragments: Any | None = None, *, cut_points: Any | None = None) Any¶
The context-parallel partition of one packed batch over
world_sizeranks.- Parameters:
cu_seqlens_global_cpu – int32
[num_docs + 1]document offsets of the whole batch, on the host (a tensor or a sequence).world_size (int) – The number of ranks of the context-parallel group.
layout (str) –
"contiguous"(equal token shares in order),"zigzag"or"zigzag_per_document"(the load-balanced layouts);fragmentsgives an explicit ragged layout instead, andcut_points(per document the2 * world_size + 1cut offsets) applies tolayout="zigzag_per_document"only.fragments (optional) – Explicit ragged layout:
fragments[rank] = [(doc, off, len), ...], or a flat[(doc, off, len, rank), ...]list.docis the document index,offits token offset within that document,lenthe token count, andrankthe owner.cut_points (optional) – Per-document
2 * world_size + 1cut offsets forlayout="zigzag_per_document".
- Returns:
CakeKDANondetKcpPartition: per rank its fragments (document, token range) of the batch; the input ofcake_kda_nondet_train_kcp_build_context()on every rank.- Return type:
CakeKDANondetKcpPartition