# mlx.nn.SinusoidalPositionalEncoding — MLX 0.0.7 documentation

Skip to main content
`Ctrl`+`K`

# mlx.nn.SinusoidalPositionalEncoding

class mlx.nn.SinusoidalPositionalEncoding(dims: int, min_freq: float = 0.0001, max_freq: float = 1, scale: Optional [ float ] = None, cos_first: bool = False, full_turns: bool = False)

  Implements sinusoidal positional encoding.

  For more details see the paper Attention Is All You Need.

  Parameters:

    - dims (int) – The dimensionality of the resulting positional embeddings.

    - min_freq (float , optional) – The minimum frequency expected. Default: `0.0001`.

    - max_freq (float , optional) – The maximum frequency expected. Default: `1`.

    - scale (float , optional) – A multiplicative scale for the embeddings. Default: `sqrt(dims//2)`.

    - cos_first (bool , optional) – If `True` embed using `[cos(x); sin(x)]` instead of the reverse. Default: `False`.

    - full_turns (bool , optional) – If `True` multiply the frequencies with \(2\pi\). Default: `False`.