Gökdeniz Gülmez
4b44434c54
Merge branch 'ml-explore:main' into adding-dpo-training
2025-02-12 11:09:58 +01:00
Awni Hannun
ec30dc3538
hunyuan finetune ( #1270 )
2025-02-11 16:49:35 -08:00
Awni Hannun
42413c5d85
fix lora timings after validation ( #1278 )
2025-02-11 16:48:55 -08:00
Awni Hannun
f8cbf159e0
fix sharding for more even number of layers ( #1276 )
2025-02-11 16:26:59 -08:00
Awni Hannun
e879ea70e1
fix generation evaluations ( #1277 )
2025-02-11 16:10:30 -08:00
Matt Clayton
3d677f0870
Add "from_draft" to GenerationResponse ( #1272 )
...
* Add from_draft field in GenerationResponse
* Cleanup
* Re-work for minimal changes, add test
* Fix comment
2025-02-11 15:41:02 -08:00
Goekdeniz-Guelmez
3716946aae
Merge branch 'adding-dpo-training' of https://github.com/Goekdeniz-Guelmez/mlx-examples into adding-dpo-training
2025-02-10 10:56:57 +01:00
Goekdeniz-Guelmez
aa932e83e3
nits
2025-02-10 10:56:50 +01:00
Gökdeniz Gülmez
16a3c7613f
Merge branch 'main' into adding-dpo-training
2025-02-10 10:55:39 +01:00
Chime Ogbuji
5865899c81
Completion only fine-tuning of instruction models with collections of HF datasets ( #1103 )
...
- Optional completion only fine-tuning with `--mask-prompt`
- Collections of Hugging Face datasets
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2025-02-09 20:12:34 -08:00
Sri Harsha Pamu
1ced1b00ca
rm temp argument ( #1267 )
2025-02-09 11:39:11 -08:00
Awni Hannun
1503bd4f55
support hunyuan 7b ( #1263 )
2025-02-08 15:46:47 -08:00
Awni Hannun
31611b62d7
Add IBM granite model ( #1265 )
...
* add granite
* add thinking option
2025-02-08 15:46:15 -08:00
Awni Hannun
6120a5f376
Faster DSv2/3 expert score computation ( #1257 )
...
* fix deepseek sharding (#1242 )
* compile and use put along axis in deep seek routing function
2025-02-07 10:24:57 -08:00
Awni Hannun
52c41b5b5a
Fix prompt cache for models without chat template ( #1250 )
...
* fix deepseek sharding (#1242 )
* fix prompt cache with no chat template
2025-02-06 11:10:58 -08:00
Gökdeniz Gülmez
67106711ce
Merge branch 'ml-explore:main' into adding-dpo-training
2025-02-06 08:16:10 +01:00
Pedro Cuenca
e2e5478da5
READMEs: fix typo in link, minor update. ( #1246 )
2025-02-04 11:52:32 -08:00
Goekdeniz-Guelmez
b1c1e1353e
nice printing the test metrics
2025-02-04 11:19:59 +01:00
Goekdeniz-Guelmez
069431bd65
adding test_ppl in testing
2025-02-04 11:18:09 +01:00
Goekdeniz-Guelmez
43f2451973
nits
2025-02-04 11:02:59 +01:00
Goekdeniz-Guelmez
c2fcb6738b
fix testing
2025-02-04 11:02:00 +01:00
Gökdeniz Gülmez
9b489a6c0c
Merge branch 'ml-explore:main' into adding-dpo-training
2025-02-04 10:45:50 +01:00
Awni Hannun
21d0ab6e8a
fix deepseek sharding ( #1242 )
2025-02-03 16:59:50 -08:00
Gökdeniz Gülmez
0989c073b0
Optimizations for mamba1 ( #1213 )
...
* added mx.einsum() operations: before: 41.293 tokens-per-sec, after: 57.822 tokens-per-sec
* Fused Operations in delta, B, C = ... :. Before: 57.822 tokens-per-sec, after: 83.890 tokens-per-sec
* Pre-computing A_log. After: 83.890 tokens-per-sec, before: 85.848 tokens-per-sec
* Update MambaBlock, Batched Input Processing, Improved Cache Handling, Pre-computed Constants, Cleaner State Management, Explicit Return Values:. Before: 82.442 tokens-per-sec, after: 129.130 tokens-per-sec.
* cleaning up and adding apple copyright to helium modelfile
* update Copyright to this year
* nits + even faster
---------
Co-authored-by: Awni Hannun <awni.hannun@gmail.com>
2025-02-03 13:36:08 -08:00
Awni Hannun
d9924d08d1
Fix no validation in lora ( #1241 )
2025-02-03 09:55:24 -08:00
Awni Hannun
9c2ef38d4d
only download local shard ( #1240 )
2025-02-02 13:58:44 -08:00
Goekdeniz-Guelmez
fbb51f651a
small fix
2025-02-01 16:08:52 +01:00
Goekdeniz-Guelmez
a03d434bb9
clean up
2025-01-31 21:37:15 +01:00
Goekdeniz-Guelmez
5998272ec2
cleaning up some namings
2025-01-31 21:27:59 +01:00
Goekdeniz-Guelmez
b379359385
small fix
2025-01-31 17:19:55 +01:00
Goekdeniz-Guelmez
b31d9cbb65
removing is-reference-free argument
2025-01-31 00:01:49 +01:00
Gökdeniz Gülmez
b3d6fc38cd
Merge branch 'ml-explore:main' into adding-dpo-training
2025-01-29 15:07:37 +01:00
Awni Hannun
e8afb59de4
better overflow correction ( #1229 )
2025-01-28 14:37:30 -08:00
Anchen
7a83077cd7
chore(mlx-lm): support text type content in messages ( #1225 )
...
* chore(mlx-lm): support text type content
* chore: optimize the messagef content processing
* nits + format
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2025-01-27 17:13:50 -08:00
Awni Hannun
f44a52e2dc
batched min p and fix spec gen sampling ( #1222 )
2025-01-27 15:40:31 -08:00
Gökdeniz Gülmez
9e5482ee74
Merge branch 'main' into adding-dpo-training
2025-01-26 17:01:37 +01:00
Gökdeniz Gülmez
77faa14ba4
adding support for kyutai's helium ( #1208 )
...
* initial commit
* adding helium into training
* Update ACKNOWLEDGMENTS.md
* nits
* nits
* fixes / nits
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2025-01-26 07:19:07 -08:00
Goekdeniz-Guelmez
557649d8da
removing tokenizer and updates
2025-01-26 15:25:27 +01:00
Goekdeniz-Guelmez
4d0e52f7c8
more metrics
2025-01-26 15:09:55 +01:00
Goekdeniz-Guelmez
0ff1289bd9
updates
2025-01-25 22:03:32 +01:00
Goekdeniz-Guelmez
86b315fdf9
nits and quality of life improvements
2025-01-24 22:40:27 +01:00
Goekdeniz-Guelmez
531c3345c6
nits
2025-01-24 18:13:05 +01:00
Goekdeniz-Guelmez
54fcd8ed63
update DPODataset and added in system field too
2025-01-24 18:11:56 +01:00
Goekdeniz-Guelmez
aefe4ba160
nits
2025-01-22 21:36:56 +01:00
Goekdeniz-Guelmez
e1d549bcd3
nits
2025-01-22 21:03:21 +01:00
Goekdeniz-Guelmez
b0ece88909
nits
2025-01-22 20:54:31 +01:00
Gökdeniz Gülmez
69a8f11f7b
Merge branch 'ml-explore:main' into adding-dpo-training
2025-01-22 14:18:24 +01:00
Awni Hannun
9a3ddc3e65
some fixes for pipeline parallel deep seek r1 ( #1216 )
2025-01-21 19:40:29 -08:00
Victor Nogueira
df1406735b
Fix dataset variable name, in datasets.py ( #1212 )
2025-01-21 14:12:43 -08:00
Goekdeniz-Guelmez
477000ec9d
removing unneeded functions
2025-01-19 01:13:17 +01:00
