flashinfer.activation

This module provides a set of activation operations for up/gate layers in transformer MLPs.

Up/Gate output activation

silu_and_mul(input[, out])	Fused SiLU and Mul operation.
gelu_tanh_and_mul(input[, out])	Fused GeLU Tanh and Mul operation.
gelu_and_mul(input[, out])	Fused GeLU and Mul operation.