mlx-examples/llms/phixtral

Phixtral

Phixtral is a Mixture of Experts (MoE) architecture inspired by Mixtral but made by combinding fine-tuned versions of Phi-2.^{1 2}

Setup

Install the dependencies:

pip install -r requirements.txt

Run

python generate.py \
  --model mlabonne/phixtral-4x2_8 \
  --prompt "write a quick sort in Python"

Run python generate.py --help to see all the options.

---

1. For more details on Phixtral, see the Hugging Face repo. ↩︎

2. For more details on Phi-2 see Microsoft's blog post and the Hugging Face repo. ↩︎