mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-06-24 09:21:18 +08:00

History

Haixing Hu 5b62270556 fix: fix issue #54 , use CPU device to load the Torch model		2023-12-11 10:54:55 +08:00
..
convert.py	fix: fix issue #54 , use CPU device to load the Torch model	2023-12-11 10:54:55 +08:00
llama.py	add copyright in source	2023-11-30 11:08:53 -08:00
README.md	📝 apply the path change to the convert cmd for consistency	2023-12-08 17:11:50 +01:00
requirements.txt	Add the Llama and Stable Diffusion examples	2023-11-29 10:38:20 -08:00
sample_prompt.txt	Add the Llama and Stable Diffusion examples	2023-11-29 10:38:20 -08:00

README.md

LLaMA

An example of generating text with LLaMA using MLX.

LLaMA is a set of open source language models from Meta AI Research¹ ranging from 7B to 65B parameters.

Setup

Install the dependencies:

pip install -r requirements.txt

Next, download and convert the model. If you do not have access to the model weights you will need to request access from Meta.

Alternatively, you can also download a select converted checkpoints from the mlx-llama community organisation on Hugging Face and skip the conversion step.

Convert the weights with:

python convert.py <path_to_torch_weights> <path_to_mlx_llama_weights.npz>

Run

Once you've converted the weights to MLX format, you can interact with the LLaMA model:

python llama.py <path_to_mlx_llama_weights.npz> <path_to_tokenizer.model> "hello"

Run python llama.py --help for more details.

Refer to the arXiv paper and blog post for more details. ↩︎