flashinfer.comm._ulysses_lowp.quant_kv_into_payload

flashinfer.comm._ulysses_lowp.quant_kv_into_payload(k: Tensor, v: Tensor, k_mean_global: Tensor, k_amax_final: Tensor, v_scale_global: Tensor, send_u8: Tensor, *, rank: int, world_size: int, enable_pdl: bool | None = None) → None

Quantize K (global grid) and V (global per-channel) into the payload.

Parameters:

  • k – BF16/FP16 CUDA K shard of shape [B, L, H, D], with the same layout and alignment requirements as Q.

  • v – BF16/FP16 CUDA V shard of shape [B, L, H, D], with the same layout and alignment requirements as Q.

  • k_mean_global – Contiguous [B, H, D] global K mean, rounded to the input dtype.

  • k_amax_final – Contiguous FP32 [B, H, slots(L, 64)] centered-K maxima after the cross-rank boundary merge.

  • v_scale_global – Contiguous FP32 [B, H, D] global per-channel V divisor, global_amax / 2.25; zero-amax channels keep scale 0 and quantize to zero.

  • send_u8 – Contiguous CUDA uint8 destination-major buffer [world_size, chunk_bytes] from the matching payload specification.

  • rank – Sequence-shard rank in [0, world_size).

  • world_size – Ulysses group size, one of 2, 4, 6 or 8; 6 is supported structurally but has not been validated.

  • enable_pdl – Whether to enable Programmatic Dependent Launch. None selects according to device support.