mirror of
https://github.com/ml-explore/mlx-examples.git
synced 2025-07-28 17:01:13 +08:00
5de7c2ac33
* Add tips on porting LLMs from HuggingFace * Add CONTRIBUTING.md to mlx-examples-llms * Refactor imports and update comment in starcoder2.py * Update llms/mlx_lm/models/starcoder2.py Co-authored-by: Awni Hannun <awni.hannun@gmail.com> * nits * nits --------- Co-authored-by: Awni Hannun <awni.hannun@gmail.com> Co-authored-by: Awni Hannun <awni@apple.com>
1.4 KiB
1.4 KiB
Contributing to MLX LM
Below are some tips to port LLMs available on Hugging Face to MLX.
Before starting checkout the general contribution guidelines.
Next, from this directory, do an editable install:
pip install -e .
Then check if the model has weights in the safetensors format. If not follow instructions to convert it.
After that, add the model file to the
mlx_lm/models
directory. You can see other examples there. We recommend starting from a model
that is similar to the model you are porting.
Make sure the name of the new model file is the same as the model_type in the
config.json, for example
starcoder2.
To determine the model layer names, we suggest either:
- Refer to the Transformers implementation if you are familiar with the codebase.
- Load the model weights and check the weight names which will tell you about the model structure.
- Look at the names of the weights by inspecting
model.safetensors.index.jsonin the Hugging Face repo.
To add LoRA support edit
mlx_lm/tuner/utils.py