12 Commits

Author	SHA1	Message	Date
Awni Hannun	2146bcd7ee	Quantize embedding / Update quantize API (#680) ... * more async eval * quantize embedding / update quantize api * more updates for quantize * update for quantize embeddings * update sd quant API * update sdxl quants * error for datasets < batch_size * async * fix config loading * fix quant * fix tests * fix req * remove lm head if tie weights is true * fix test	2024-04-18 18:16:10 -07:00
Awni Hannun	9c5554d8ee	Use async eval (#670) ... * Use async eval * bump * bump * remove workaround for bfloat cumsum	2024-04-11 13:18:23 -07:00
devonthomas35	9f472dc985	Update transformers for ⌘-R+ (#668)	2024-04-11 07:28:12 -07:00
Awni Hannun	b8a348c1b8	Switch to fast RMS/LN Norm (#603) ... * use nn.RMSNorm, use sdpa, cleanup * bump mlx versions * minor update * use fast layer norm * version bump * update requirement for whisper * update requirement for gguf	2024-03-23 07:13:51 -07:00
madroid	b0bcd86a40	Support for OpenAI’s fine-tuning dataset format (#548) ... * LoRA: move load_dataset to tuner/datasets.py file * LoRA: support OpenAI chat format datasets see https://platform.openai.com/docs/guides/fine-tuning/example-format * LoRA: support OpenAI completion format datasets * LoRA: formatting dataset timing to reduce memory footprint * Refactor dataset item access in PromptCompletionDataset * Update mlx_lm/LORA.md * Update mlx_lm/LORA.md * check Unsupported data format * add tests, fine-tune doc * add tests, fine-tune doc * add jinja2 for chat template * nits in readme * nits in readme --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-03-19 16:45:46 -07:00
Awni Hannun	8b05bb6d18	[mlx-lm] Use sdpa in llama / mistral model (#515) ... * use sdpa * update a few more models * version * fix stablelm type	2024-03-07 17:41:23 -08:00
Awni Hannun	f24edfa9dc	[mlx-lm] Add precompiled normalizations (#451) ... * add precompiled normalizations * nits	2024-02-22 12:40:55 -08:00
Awni Hannun	ab9172baac	Gemma support (#474) ... * gemma support * format * lora support for gemma	2024-02-21 08:47:13 -08:00
Awni Hannun	8fd953ee2b	Support for slerp merging models (#455) ... * support for slerp merging models * docs * update docs * format'	2024-02-19 20:37:15 -08:00
Awni Hannun	f45a1ab83c	Update a few examples to use compile (#420) ... * update a few examples to use compile * update mnist * add compile to vae and rename some stuff for simplicity * update reqs * use state in eval * GCN example with RNG + dropout * add a bit of prefetching	2024-02-08 13:00:41 -08:00
Anchen	30be4c4734	refactor(qwen): moving qwen into mlx-lm (#312) ... * refactor(qwen): moving qwen into mlx-lm * chore: update doc * chore: fix type hint * add qwen model support in convert * chore: fix doc * chore: only load model in quantize_model * chore: make the convert script only copy tokenizer files instead of load it and save * chore: update docstring * chore: remove unnecessary try catch * chore: clean up for tokenizer and update transformers 4.37 * nits in README --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-01-22 15:00:07 -08:00
Awni Hannun	c6440416a2	Mlx llm package (#301) ... * fix converter * add recursive files * remove gitignore * remove gitignore * add packages properly * read me update * remove dup readme * relative * fix convert * fix community name * fix url * version	2024-01-12 10:25:56 -08:00