9 Commits

Author	SHA1	Message	Date
Awni Hannun	ab9172baac	Gemma support (#474) ... * gemma support * format * lora support for gemma	2024-02-21 08:47:13 -08:00
Madroid Ma	726b1ddec0	fix: check LoRA layers number error (#446)	2024-02-16 06:03:33 -08:00
Madroid Ma	954aa50c54	LoRA: Improve validation error for LoRA layer count exceeding model layer (#427) ... * LoRA: Improve validation error for LoRA layer count exceeding model layer This commit enhances the error handling when the specified LoRA layer count exceeds the total number of layers in the model. It clarifies the error message to provide actionable feedback for users, guiding them to adjust their input parameters accordingly. * format + nits --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-02-13 06:56:27 -08:00
Awni Hannun	d4666615bb	Lazy import + refactor Lora layer addition (#426) ... * lazy model import in mlx_lm * change lora loading * fix olmo lora * remove a bunch of unused stuff from plamo * move phixtral to mlx-lm and out of llms/	2024-02-12 10:51:02 -08:00
Anchen	0a49ba0697	fix(mlx-lm): apply lora layer doesn't update the lora weights (#396)	2024-01-31 11:51:26 -08:00
Anchen	614de6652f	chore(mlx-lm): add reset lora layers helper (#377) ... * chore(mlx-lm): add reset lora layers helper * chore: rename the func * chore: update docstring * Update llms/mlx_lm/tuner/utils.py Co-authored-by: Awni Hannun <awni.hannun@gmail.com> --------- Co-authored-by: Awni Hannun <awni.hannun@gmail.com>	2024-01-29 20:54:49 -08:00
Anchen	854ad8747a	feat(mlx-lm): add de-quant for fuse.py (#365) ... * feat(mlx-lm): add de-quant for fuse * chore: disable quant in to linear when de-quant enabled * chore: add better error handling for adapter file not found	2024-01-25 18:59:32 -08:00
Anchen	ab91ac1075	chore(mlx-lm): add load model with adapter and fix bug in sample (#360) ... * chore: add load model with adapter support and fix bug in sample * chore: ignore temp during calculating prob in sample	2024-01-23 19:47:39 -08:00
Anchen	362e88a744	feat: move lora into mlx-lm (#337) ... * feat: Add lora and qlora training to mlx-lm --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-01-23 08:44:37 -08:00