flashinfer.activation

This module provides a set of activation operations for up/gate layers in transformer MLPs.

Up/Gate output activation

silu_and_mul(input[, out, enable_pdl])	Fused SiLU and Mul operation.
gelu_tanh_and_mul(input[, out, enable_pdl])	Fused GeLU Tanh and Mul operation.
gelu_and_mul(input[, out, enable_pdl])	Fused GeLU and Mul operation.
silu_and_mul_scaled_nvfp4_experts_quantize(a, ...)	Fused SiLU + mul + per-expert NVFP4 quantization with a per-row mask.