Awni Hannun
b8a348c1b8
Switch to fast RMS/LN Norm ( #603 )
...
* use nn.RMSNorm, use sdpa, cleanup
* bump mlx versions
* minor update
* use fast layer norm
* version bump
* update requirement for whisper
* update requirement for gguf
2024-03-23 07:13:51 -07:00
Sergey Shumov
15ecf692b9
Bug fix in lora.py ( #468 )
2024-02-20 12:53:30 -08:00
Awni Hannun
8fd953ee2b
Support for slerp merging models ( #455 )
...
* support for slerp merging models
* docs
* update docs
* format'
2024-02-19 20:37:15 -08:00
Ovid
8c9148a8fd
Make it easier to know in which file we have bad JSON data ( #458 )
...
* Make it easier to know which file we have bad JSON data in.
* Use a loop rather than repeat code sections.
I previously had these as separate cut-n-drooled sections of code. This change makes it a clean loop.
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
* Small fix to previous code suggestion to restore a missing variable.
---------
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
2024-02-19 20:11:45 -08:00
ZHAOKAI WANG
0340113e02
BUG FIX: Decoding results in garbled text when multiple tokens represent a single character (e.g., Chinese). ( #398 )
...
* Decoding results in garbled text when multiple tokens represent a single character (e.g., Chinese).
* Decoding results in garbled text when multiple tokens represent a single character (e.g., Chinese).
2024-01-31 19:27:29 -08:00
Anchen
1415595409
chore(lora): support mixtral in lora example ( #343 )
2024-01-20 06:07:45 -08:00
Zheng Qu
d8680a89f9
Add argument --save-every N to lora.py for saving model regularly ( #310 )
2024-01-16 20:03:33 -08:00
Yousif
7575125d5d
Added lora support for Phi-2 ( #302 )
...
* Added lora support for Phi-2
* Added Phi-2 support in fuse and convert
* format + readme
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2024-01-12 13:45:30 -08:00
Anjor Kanekar
e74889d0fa
prompt parameter ( #291 )
2024-01-11 06:04:57 -08:00
Awni Hannun
80d18671ad
[Lora] Fix generate ( #282 )
...
* fix generate
* update readme, fix test, better default
* nits
* typo
2024-01-10 16:13:06 -08:00
Awni Hannun
841c8f7b30
fix max tokens ( #275 )
2024-01-09 21:41:12 -08:00
Anchen
7cfda327fd
fix(lora): tokenizer return incompatible mx array ( #271 )
...
* fix(lora): tokenizer return incompatible encodeing mx array
* add readme nit
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2024-01-09 19:46:38 -08:00
Awni Hannun
7b258f33ac
Move lora example to use the same model format / conversion as hf_llm ( #252 )
...
* huffing face the lora example to allow more models
* fixes
* comments
* more readme nits
* fusion + works better for qlora
* nits'
* comments
2024-01-09 11:14:52 -08:00
Awni Hannun
485fb9ac0f
quantize linear ( #250 )
2024-01-07 18:48:59 -08:00
Lawrence Wu
37856f70a8
add numpy as a requirement to run lora.py ( #238 )
...
* add numpy as a requirement to run lora.py
* removed unused imports
2024-01-05 16:16:28 -08:00
Awni Hannun
37b41cec60
Qlora ( #219 )
...
qlora
2024-01-04 21:05:59 -08:00
Todsaporn Banjerdkit
7ae445f6c7
feat: add mistral tps ( #173 )
...
* feat: add mistral tps
* eval params before timing + format
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2023-12-22 07:55:57 -08:00
wyanzhao
22620de3ee
1. Add user warning for sequences over 2048 tokens in iterate_batches. ( #166 )
2023-12-21 06:29:31 -08:00
Awni Hannun
27c0a8c002
Add llms subdir + update README ( #145 )
...
* add llms subdir + update README
* nits
* use same pre-commit as mlx
* update readmes a bit
* format
2023-12-20 10:22:25 -08:00
Awni Hannun
1e7f4a5921
fix use for llama 2 from meta ( #144 )
2023-12-18 19:33:17 -08:00
Awni Hannun
84f02ef58b
use lower precision base weights
2023-12-15 10:29:42 -08:00
Awni Hannun
d108c558fc
more nits
2023-12-15 10:06:14 -08:00
Awni Hannun
985f413f99
custom data with lora
2023-12-15 09:56:10 -08:00
Awni Hannun
98f4346c81
black format
2023-12-09 14:15:25 -08:00
Awni Hannun
b8332a1e66
generalize lora finetuning for llama and mistral
2023-12-09 14:13:55 -08:00
Awni Hannun
31bc57c4ff
add copyright in source
2023-11-30 11:08:53 -08:00
Awni Hannun
5d6353aab7
lora
2023-11-29 14:14:11 -08:00
