4 Commits

Author	SHA1	Message	Date
Awni Hannun	39084e81c2	Some improvements to LoRA (#528) ... * set cache_limit * remove set cache_limit * cleanup * add gradient checkpointing * fix sort * mokey patch call for checkpoint * fix example config	2024-03-12 20:02:03 -07:00
Chime Ogbuji	e56d9015ef	LoRA on all linear transformer block layers (#546) ... * Add --lora-all-linear option to apply LoRa to all linear transfer block layers * Moved to YAML config and added specification of rank & alpha * nits in conifg, more tests * nit * run tests for prs --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-12 07:37:40 -07:00
Chime Ogbuji	8c2cf665ed	YAML configuration for mlx_lm.lora (#503) ... * Convert mlx_lm.lora to use YAML configuration * pre-commit run fixes * Fix loading of config file * Remove invalid YAML from doc * Update command-line options and YAML parameter overriding, per feedback in #503 * Minor wording change * Positional argument * Moved config to a (-c/--config) flag * Removed CLI option defaults (since CLI options take precedence and their defaults are in CONFIG_DEFAULTS) * pre-commit format updates * Fix handling of CLI option defaults * Prevent None values of unspecified CLI options from overwriting values from CONFIG_DEFAULTS * nits --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-08 07:57:52 -08:00
Awni Hannun	8fd953ee2b	Support for slerp merging models (#455) ... * support for slerp merging models * docs * update docs * format'	2024-02-19 20:37:15 -08:00