7 Commits

Author	SHA1	Message	Date
Angelos Katharopoulos	9f671228cd	Block sparse MM MoEs (#782) ... - Adds SwitchLinear - Adds QuantizedSwitchLinear	2024-05-21 15:58:08 -07:00
Awni Hannun	bbfcc103d7	cast around lora adapters (#613)	2024-03-24 19:34:51 -07:00
Chime Ogbuji	e56d9015ef	LoRA on all linear transformer block layers (#546) ... * Add --lora-all-linear option to apply LoRa to all linear transfer block layers * Moved to YAML config and added specification of rank & alpha * nits in conifg, more tests * nit * run tests for prs --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-12 07:37:40 -07:00
Awni Hannun	ad3cf5ed98	dropout 0 as default (#549)	2024-03-08 13:07:10 -08:00
Anchen	854ad8747a	feat(mlx-lm): add de-quant for fuse.py (#365) ... * feat(mlx-lm): add de-quant for fuse * chore: disable quant in to linear when de-quant enabled * chore: add better error handling for adapter file not found	2024-01-25 18:59:32 -08:00
Anchen	b1dec281b3	feat(mlx-lm): add lora hypeparameters in lora layer (#366) ... * feat(mlx-lm): add lora hypeparameters in lora layer * chore: address comments	2024-01-24 08:11:25 -08:00
Anchen	362e88a744	feat: move lora into mlx-lm (#337) ... * feat: Add lora and qlora training to mlx-lm --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-01-23 08:44:37 -08:00