flashinfer.activation

This module provides a set of activation operations for up/gate layers in transformer MLPs.

Up/Gate output activation

silu_and_mul(input[, out, enable_pdl])	Fused SiLU and Mul operation.
gelu_tanh_and_mul(input[, out, enable_pdl])	Fused GeLU Tanh and Mul operation.
gelu_and_mul(input[, out, enable_pdl])	Fused GeLU and Mul operation.
silu_and_mul_scaled_nvfp4_experts_quantize(a, ...)	Silu and multiply and quantize batched input tensor to NVFP4 format with mask.