mlx-examples/llms/qwen/README.md

Qwen

Qwen (通义千问) is a language model developed by Alibaba Cloud.¹ The architecture of Qwen is similar to Llama except for the bias in the attention layers.

Setup

First download and convert the model with:

python convert.py

The script downloads the model from Hugging Face. The default model is Qwen/Qwen-1_8B. Check out the Hugging Face page to see a list of available models.

The conversion script will make the weights.npz and config.json files in the working directory.

Generate

To generate text with the default prompt:

python qwen.py

If you change the model, make sure to pass the corresponding tokenizer. E.g., for Qwen 7B use:

python qwen.py --tokenizer  Qwen/Qwen-7B

To see a list of options, run:

python qwen.py --help

---

1. For more details on the model see the official repo of Qwen and the Hugging Face. ↩︎