Support for slerp merging models (#455)

* support for slerp merging models

* docs

* update docs

* format'
This commit is contained in:
Awni Hannun
2024-02-19 20:37:15 -08:00
committed by GitHub
parent 8c9148a8fd
commit 8fd953ee2b
16 changed files with 329 additions and 15 deletions

View File

@@ -14,9 +14,11 @@ pip install mlx-lm
conda install -c conda-forge mlx-lm
```
The `mlx-lm` package also supports LoRA and QLoRA fine-tuning. For more details
on this see the [LoRA
documentation](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/LORA.md).
The `mlx-lm` package also has:
- [LoRA and QLoRA fine-tuning](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/LORA.md)
- [Merging models](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/MERGE.md)
- [HTTP model serving](https://github.com/ml-explore/mlx-examples/blob/main/llms/mlx_lm/SERVER.md)
### Python API
@@ -25,7 +27,7 @@ You can use `mlx-lm` as a module:
```python
from mlx_lm import load, generate
model, tokenizer = load("mistralai/Mistral-7B-v0.1")
model, tokenizer = load("mistralai/Mistral-7B-Instruct-v0.1")
response = generate(model, tokenizer, prompt="hello", verbose=True)
```
@@ -44,7 +46,7 @@ You can convert models in the Python API with:
```python
from mlx_lm import convert
upload_repo = "mlx-community/My-Mistral-7B-v0.1-4bit"
upload_repo = "mistralai/Mistral-7B-Instruct-v0.1"
convert("mistralai/Mistral-7B-v0.1", quantize=True, upload_repo=upload_repo)
```
@@ -64,7 +66,7 @@ To see a description of all the arguments you can do:
You can also use `mlx-lm` from the command line with:
```
python -m mlx_lm.generate --model mistralai/Mistral-7B-v0.1 --prompt "hello"
python -m mlx_lm.generate --model mistralai/Mistral-7B-Instruct-v0.1 --prompt "hello"
```
This will download a Mistral 7B model from the Hugging Face Hub and generate
@@ -79,7 +81,7 @@ python -m mlx_lm.generate --help
To quantize a model from the command line run:
```
python -m mlx_lm.convert --hf-path mistralai/Mistral-7B-v0.1 -q
python -m mlx_lm.convert --hf-path mistralai/Mistral-7B-Instruct-v0.1 -q
```
For more options run: