mlx.nn.gelu_approx

class mlx.nn.gelu_approx(x)

An approximation to Gaussian Error Linear Unit.

See gelu() for the exact computation.

This function approximates gelu with a maximum absolute error \(< 0.0003\) in the range \([-6, 6]\) using the following

\[x = x \sigma\left(1.60033 x \left(1 + 0.0433603 x^2\right)\right)\]

where \(\sigma(\cdot)\) is the logistic sigmoid.