zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-09-01 12:49:50 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

use official HF for mixtral

Browse Source
This commit is contained in:
Awni Hannun
2023-12-14 15:30:32 -08:00
parent 09fff84a85
commit 078fed3d8d
4 changed files with 54 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
mixtral/README.md
View File

@@ -17,36 +17,28 @@ brew install git-lfs
Download the models from Hugging Face:
```
git-lfs clone https://huggingface.co/someone13574/mixtral-8x7b-32kseqlen
```
After that's done, combine the files:
```
cd mixtral-8x7b-32kseqlen/
cat consolidated.00.pth-split0 consolidated.00.pth-split1 consolidated.00.pth-split2 consolidated.00.pth-split3 consolidated.00.pth-split4 consolidated.00.pth-split5 consolidated.00.pth-split6 consolidated.00.pth-split7 consolidated.00.pth-split8 consolidated.00.pth-split9 consolidated.00.pth-split10 > consolidated.00.pth
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/mistralai/Mixtral-8x7B-v0.1/
cd Mixtral-8x7B-v0.1/ && \
git lfs pull --include "consolidated.*.pt" && \
git lfs pull --include "tokenizer.model"
```
Now from `mlx-exmaples/mixtral` convert and save the weights as NumPy arrays so
MLX can read them:
```
python convert.py --model_path mixtral-8x7b-32kseqlen/
python convert.py --model_path Mixtral-8x7B-v0.1/
```
The conversion script will save the converted weights in the same location.
After that's done, if you want to clean some stuff up:
```
rm mixtral-8x7b-32kseqlen/*.pth*
```
### Generate
As easy as:
```
python mixtral.py --model_path mixtral-8x7b-32kseqlen/
python mixtral.py --model_path Mixtral-8x7B-v0.1/
```
[^mixtral]: Refer to Mistral's [blog post](https://mistral.ai/news/mixtral-of-experts/) for more details.
[^mixtral]: Refer to Mistral's [blog
post](https://mistral.ai/news/mixtral-of-experts/) for more details.