flashinfer.quantization.silu_and_mul_nvfp4_quantize

flashinfer.quantization.silu_and_mul_nvfp4_quantize(input: ~torch.Tensor, global_scale: ~torch.Tensor, sf_vec_size: int = 16, is_sf_swizzled_layout: bool = True, is_sf_8x4_layout: bool = False, enable_pdl: bool | None = None, nvfp4_4over6: ~flashinfer.quantization.nvfp4_quantization_utils.NVFP44Over6Config | None = <nvfp4_4over6 unset: read FLASHINFER_NVFP4_4OVER6*>) → Tuple[Tensor, Tensor]

Apply SwiGLU and NVFP4 quantization in one CuTe-DSL kernel.

Computes silu(input[..., :K]) * input[..., K:] before quantization. Requires CuTe-DSL and an SM100+ GPU.

Parameters:

• input (torch.Tensor) – Contiguous fp16/bf16 tensor of shape [..., 2K]. Leading dimensions are flattened into M.

• global_scale (torch.Tensor) – Float32 scale of shape [1], typically (448 * 6) / silu_and_mul(input).abs().max().

• sf_vec_size (int) – Scale vector size. Only 16 is supported.

• is_sf_swizzled_layout (bool) – Use a swizzled scale layout. Defaults to True.

• is_sf_8x4_layout (bool) – Use the 8x4 rather than 128x4 swizzled layout.

• enable_pdl (bool, optional) – Enable Programmatic Dependent Launch. Auto-detected when None.

• nvfp4_4over6 (NVFP44Over6Config or None) –

  NVFP4 “4over6” scale-candidate search. Requires fp16 / bf16 input.

  • omitted (the default): read the legacy FLASHINFER_NVFP4_4OVER6* environment variables.

  • None: 4over6 off. The environment is ignored.

  • NVFP44Over6Config: on with exactly that recipe. The environment is ignored.

Returns:

Packed FP4 values of shape [M, K/2] and their scale factors.

Return type:

Tuple[torch.Tensor, torch.Tensor]