mlx-examples/llms/mlx_lm/tuner
Prince Canuma d661440dbb
Add support for qwen2moe (#640)
* add sparsemoe block and update decoder logic

* update file name to match HF

* update name

* Code formatting

* update gates calculation

* add support for Qwen2MoE.

* fix pytest

* code formatting and fix missing comma in utils

* Remove decoder sparse step.

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>

* remove gate layer anti-quantisation

* remove unused argument

---------

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>
2024-04-02 11:33:29 -07:00
..
__init__.py feat: move lora into mlx-lm (#337) 2024-01-23 08:44:37 -08:00
datasets.py Support for OpenAI’s fine-tuning dataset format (#548) 2024-03-19 16:45:46 -07:00
lora.py cast around lora adapters (#613) 2024-03-24 19:34:51 -07:00
trainer.py chore: fix loar for moe model (#608) 2024-03-23 07:22:11 -07:00
utils.py Add support for qwen2moe (#640) 2024-04-02 11:33:29 -07:00