flashinfer.comm.CakeFp4AllReduceFusedNormResult

class flashinfer.comm.CakeFp4AllReduceFusedNormResult(output: CakeFp4AllReduceFusedNormOutput, residual_out: Tensor, out: Tensor | None = None, values: Tensor | None = None, scales: Tensor | None = None)

Outputs of one fused call.

residual_out is bf16(sum + residual) ([rows, 6144]). With the bfloat16 output out holds y; with the NVFP4 output values holds the packed E2M1 pairs (uint8 [rows, 3072], element 2i in the low nibble) and scales the 128x4-swizzled E4M3 block scales (uint8, ceil(rows / 128) * 128 * 384 bytes); with the MXFP8 output values holds E4M3 bytes ([rows, 6144], torch.float8_e4m3fn) and scales the swizzled UE8M0 scales (uint8, ceil(rows / 128) * 128 * 192 bytes) – the layouts fp4_quantize / mxfp8_quantize produce for y. Scale bytes of padding rows are left untouched (zero when allocated here).

__init__(output: CakeFp4AllReduceFusedNormOutput, residual_out: Tensor, out: Tensor | None = None, values: Tensor | None = None, scales: Tensor | None = None) → None

Methods

__init__(output, residual_out[, out, ...])

Attributes

out
scales
values
output
residual_out