zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-09-01 04:14:38 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Example reading directly from gguf file (#222)

Browse Source 
* Draft of tiny llama from gguf

* Transpose all

* No transposition with new layout

* Read config from gguf

* Create tokenizer from gguf

* move gguf and update to be similar to hf_llm

* change model to HF style + updates to REAMDE

* nits in REAMDE

* nit readme

* only use mlx for metadata

* fix eos/bos tokenizer

* fix tokenization

* quantization runs

* 8-bit works

* tokenizer fix

* bump mlx version

---------

Co-authored-by: Juarez Bochi <juarez.bochi@grammarly.com>
Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
Juarez Bochi
2024-01-23 18:41:54 -05:00
committed by GitHub
parent 40b61c1719
commit f5b80c95fb
5 changed files with 536 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
llms/gguf_llm/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
mlx>=0.0.11
numpy
protobuf==3.20.0
sentencepiece