flashinfer.norm.layernorm

flashinfer.norm.layernorm(input: Tensor, gemma: Tensor, beta: Tensor, eps: float = 1e-06) → Tensor

Layer normalization.

Parameters:
  • input (torch.Tensor) – Input tensor, shape (batch_size, hidden_size). Need to be bfloat16. The last dimension must be contiguous. The default CuTe DSL backend requires a row stride that is a multiple of its vector width and a 16-byte-aligned data pointer; the CUDA backend (FLASHINFER_USE_CUDA_NORM=1, or when the CuTe DSL is unavailable) accepts any row stride.

  • gemma (torch.Tensor) – Gemma tensor, shape (hidden_size,). Need to be float32.

  • beta (torch.Tensor) – Beta tensor, shape (hidden_size,). Need to be float32.

  • eps (float) – Epsilon for numerical stability.

Returns:

output – Layer Normalized tensor, shape (batch_size, hidden_size). Same dtype as input.

Return type:

torch.Tensor