flashinfer.comm._ulysses_lowp.quant_and_pack

flashinfer.comm._ulysses_lowp.quant_and_pack(q: Tensor, k: Tensor, v: Tensor, stats: V2GStats, *, out: Tensor | None = None, enable_pdl: bool | None = None) → Tensor

Quantize and pack using the finalized global-grid Q/K amax and V scale.

Parameters:

  • q – BF16/FP16 CUDA Q shard of shape [B, L, H, D]; D is 64 or 128. The last dimension must be dense and every row 16-byte aligned.

  • k – BF16/FP16 CUDA K shard of shape [B, L, H, D], with the same layout and alignment requirements as Q.

  • v – BF16/FP16 CUDA V shard of shape [B, L, H, D], with the same layout and alignment requirements as Q.

  • stats – Finalized quantization inputs returned by finalize_stats.

  • out – Optional contiguous uint8 [world_size, chunk_bytes] output on the input device.

  • enable_pdl – Whether to enable Programmatic Dependent Launch. None selects according to device support.

Returns:

Contiguous uint8 CUDA payload of shape [world_size, chunk_bytes].