flashinfer.gated_act_mxfp8.silu_and_mul_mxfp8_quantize

flashinfer.gated_act_mxfp8.silu_and_mul_mxfp8_quantize(gated_input: Tensor, *, rowwise: bool = True, colwise: bool = False) → Tuple[Tensor, Tensor, Tensor, Tensor]

Apply SwiGLU and emit RCEIL MXFP8 rowwise and/or colwise outputs.

gated_input has shape [M, 2K] and stores gate values followed by up values. The returned tuple is ordered as (row_output, col_output, row_scales, col_scales). A disabled orientation is represented by zero-sized tensors on the input device.

Parameters:

  • gated_input (torch.Tensor) – Contiguous BF16 CUDA tensor with shape [M, 2K].

  • rowwise (bool, optional) – Whether to emit rowwise MXFP8 values and scales.

  • colwise (bool, optional) – Whether to emit colwise MXFP8 values and scales.

Returns:

Rowwise values, colwise values, rowwise scales, and colwise scales. Outputs for disabled orientations are zero-sized tensors.

Return type:

tuple of torch.Tensor