# mlx.nn.GELU — MLX 0.0.7 documentation

Skip to main content
`Ctrl`+`K`

# mlx.nn.GELU

class mlx.nn.GELU(approx='none')

  Applies the Gaussian Error Linear Units.

  \[\textrm{GELU}(x) = x * \Phi(x)\]

  where \(\Phi(x)\) is the Gaussian CDF.

  However, if `approx` is set to ‘precise’ or ‘fast’ it applies

  \[\begin{split}\textrm{GELUApprox}(x) &= x * \sigma\left(1.60033 * x \left(1 + 0.0433603 * x^2\right)\right) \\ \textrm{GELUFast}(x) &= x * \sigma\left(1.773 * x\right)\end{split}\]

  respectively.

  See `gelu()`, `gelu_approx()` and `gelu_fast_approx()` for the functional equivalents and information regarding error bounds.

  Parameters:

    approx ('none' | 'precise' | 'fast') – Which approximation to gelu to use if any.