zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-06-24 17:31:18 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
423 Commits 10 Branches 0 Tags 80 MiB
92430df0a0
Commit Graph

2 Commits

Author SHA1 Message Date
Awni Hannun
92430df0a0
Fix lora for qwen moe (#743) 
* fix lora for qwen moe

* use max seq length in test as well
 2024-05-02 21:55:09 -07:00
Prince Canuma
d661440dbb
Add support for qwen2moe (#640) 
* add sparsemoe block and update decoder logic

* update file name to match HF

* update name

* Code formatting

* update gates calculation

* add support for Qwen2MoE.

* fix pytest

* code formatting and fix missing comma in utils

* Remove decoder sparse step.

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>

* remove gate layer anti-quantisation

* remove unused argument

---------

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>
 2024-04-02 11:33:29 -07:00