alexC-nonsense4k 42458914c8 support dora finetune in mlx-examples/llms/mlx_lm (#779) ...

* support dora finetune

* solve problems in lora.py and tuner.utils.py

* add use_dora (bool) in functions of load adapters

* delete all unsupported quantization code and fix all the calculate problems in mlx_lm/tuner/dora.py

* Using stop_gradient to prevent gradients from flowing through ‘norm’ during backpropagation

* set DEFAULT_USE_DORA in mlx_lm/generate.py

* add annotation for all the use_dora

* mlx_lm/fuse.py support fuse dora layers and fix a bug of to_linear() in mlx_lm/tuner/dora.py

* simplify code of juding type of a fused layer in mlx_lm/fuse.py

* add use_dora in mlx_lm/fuse.py when apply_lora_layers()

* style + nits

* style + nits

* more updates

---------

Co-authored-by: chenyifei08 <chenyifei08@baidu.com>
Co-authored-by: Awni Hannun <awni@apple.com>

2024-05-16 08:21:26 -07:00

..

lora_config.yaml

support dora finetune in mlx-examples/llms/mlx_lm (#779)

2024-05-16 08:21:26 -07:00

merge_config.yaml

Support for slerp merging models (#455)

2024-02-19 20:37:15 -08:00