Prince Canuma d661440dbb Add support for qwen2moe (#640) ...

* add sparsemoe block and update decoder logic

* update file name to match HF

* update name

* Code formatting

* update gates calculation

* add support for Qwen2MoE.

* fix pytest

* code formatting and fix missing comma in utils

* Remove decoder sparse step.

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>

* remove gate layer anti-quantisation

* remove unused argument

---------

Co-authored-by: bozheng-hit <dsoul0621@gmail.com>

2024-04-02 11:33:29 -07:00

..

test_datsets.py

Support for OpenAI’s fine-tuning dataset format (#548)

2024-03-19 16:45:46 -07:00

test_gguf.py

fix(mlx-lm): type hints in gguf.py (#621)

2024-03-26 07:56:01 -07:00

test_lora.py

Configurable LR schedulers (#604)

2024-03-29 13:41:10 -07:00

test_models.py

Add support for qwen2moe (#640)

2024-04-02 11:33:29 -07:00

test_sample_utils.py

fix(mlx-lm): type hints in gguf.py (#621)

2024-03-26 07:56:01 -07:00

test_utils.py

LoRA on all linear transformer block layers (#546)

2024-03-12 07:37:40 -07:00