7cdd1b69ac
* add a few tests for mlx lm * add a few tests for mlx lm * add a few tests for mlx lm * more tests / cleanup
1.7 KiB
Contributing to MLX LM
Below are some tips to port LLMs available on Hugging Face to MLX.
Before starting checkout the general contribution guidelines.
Next, from this directory, do an editable install:
pip install -e .
Then check if the model has weights in the safetensors format. If not follow instructions to convert it.
After that, add the model file to the
mlx_lm/models
directory. You can see other examples there. We recommend starting from a model
that is similar to the model you are porting.
Make sure the name of the new model file is the same as the model_type in the
config.json, for example
starcoder2.
To determine the model layer names, we suggest either:
- Refer to the Transformers implementation if you are familiar with the codebase.
- Load the model weights and check the weight names which will tell you about the model structure.
- Look at the names of the weights by inspecting
model.safetensors.index.jsonin the Hugging Face repo.
To add LoRA support edit
mlx_lm/tuner/utils.py
Finally, add a test for the new modle type to the model tests.
From the llms/ directory, you can run the tests with:
python -m unittest discover tests/