10 Commits

Author	SHA1	Message	Date
Gökdeniz Gülmez	2c1c9e9024	MiniCPM implementation (#685) ... * Added support for the MiniCPM architecture * Added support for the MiniCPM architecture * Updated utils.py and LORA.md * Updated utils.py and LORA.md * Update implementation details for MiniCPM architecture * Cleaning up * fixed the missing lm.head layer problem * Refactor Model class to dynamically handle tied and untied word embeddings * Quick update * added a dynamic rope scaling base calucaltion * Added support for the MiniCPM architecture * Added support for the MiniCPM architecture * Updated utils.py and LORA.md * Updated utils.py and LORA.md * Update implementation details for MiniCPM architecture * Cleaning up * fixed the missing lm.head layer problem * Refactor Model class to dynamically handle tied and untied word embeddings * added a dynamic rope scaling base calucaltion * quick fix and clean up * clean up again * removed the MiniCPMNorm class as its not used * forgot something, sorry * format * version bump --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-04-25 15:29:28 -07:00
Phúc H. Lê Khắc	35206806ac	Create executables for generate, lora, server, merge, convert (#682) ... * feat: create executables mlx_lm.<cmd> * nits in docs --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-04-16 16:08:49 -07:00
Awni Hannun	2bd64b78cf	Save lora config (#636) ... * lora config * comments * version bump	2024-04-02 13:52:53 -07:00
Anchen	fe96ef342f	feat(mlx-lm): export the GGUF (fp16) format model weights from fuse.py (#555) ... * wip * wip * feat: convert mlx model to gguf f16 * chore: conver norm layer to float32 to avoid overflow issue * chore: add support for mixtral * chore: clean up * chore: remove unused import statement * chore: clean up weight name mapping * version and readme * actual version bump --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-21 10:34:11 -07:00
madroid	b0bcd86a40	Support for OpenAI’s fine-tuning dataset format (#548) ... * LoRA: move load_dataset to tuner/datasets.py file * LoRA: support OpenAI chat format datasets see https://platform.openai.com/docs/guides/fine-tuning/example-format * LoRA: support OpenAI completion format datasets * LoRA: formatting dataset timing to reduce memory footprint * Refactor dataset item access in PromptCompletionDataset * Update mlx_lm/LORA.md * Update mlx_lm/LORA.md * check Unsupported data format * add tests, fine-tune doc * add tests, fine-tune doc * add jinja2 for chat template * nits in readme * nits in readme --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-19 16:45:46 -07:00
Awni Hannun	e4b19bb9e1	Make attention faster for a some models (#574) ... * make attention faster for a couple models * remove unused generation flags * add comment on lora * include text files as well	2024-03-14 21:35:54 -07:00
Chime Ogbuji	8c2cf665ed	YAML configuration for mlx_lm.lora (#503) ... * Convert mlx_lm.lora to use YAML configuration * pre-commit run fixes * Fix loading of config file * Remove invalid YAML from doc * Update command-line options and YAML parameter overriding, per feedback in #503 * Minor wording change * Positional argument * Moved config to a (-c/--config) flag * Removed CLI option defaults (since CLI options take precedence and their defaults are in CONFIG_DEFAULTS) * pre-commit format updates * Fix handling of CLI option defaults * Prevent None values of unspecified CLI options from overwriting values from CONFIG_DEFAULTS * nits --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-08 07:57:52 -08:00
Anchen	13794a05da	chore(mlx-lm): add adapter support in generate.py (#494) ... * chore(mlx-lm): add adapter support in generate.py * chore: remove generate from lora.py and raise error to let user use mlx_lm.generate instead	2024-02-28 07:49:25 -08:00
Awni Hannun	8fd953ee2b	Support for slerp merging models (#455) ... * support for slerp merging models * docs * update docs * format'	2024-02-19 20:37:15 -08:00
Anchen	362e88a744	feat: move lora into mlx-lm (#337) ... * feat: Add lora and qlora training to mlx-lm --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-01-23 08:44:37 -08:00