Awni Hannun
81318ad4a8
Port of phi3small ( #794 )
...
* start port of phi3small
* fix phi3
* use block sparsity
* compile activation
* nits in readme / mlx lm version
2024-05-31 12:54:14 -07:00
Awni Hannun
ca7ce60c91
Rename block sparse to gather ( #793 )
...
* rename block sparse to gather
* pin mlx version
2024-05-23 19:47:35 -07:00
Awni Hannun
9fc6efbd90
version bump + some fixes ( #792 )
2024-05-21 20:09:35 -07:00
Awni Hannun
2146bcd7ee
Quantize embedding / Update quantize API ( #680 )
...
* more async eval
* quantize embedding / update quantize api
* more updates for quantize
* update for quantize embeddings
* update sd quant API
* update sdxl quants
* error for datasets < batch_size
* async
* fix config loading
* fix quant
* fix tests
* fix req
* remove lm head if tie weights is true
* fix test
2024-04-18 18:16:10 -07:00
Awni Hannun
9c5554d8ee
Use async eval ( #670 )
...
* Use async eval
* bump
* bump
* remove workaround for bfloat cumsum
2024-04-11 13:18:23 -07:00
devonthomas35
9f472dc985
Update transformers for ⌘-R+ ( #668 )
2024-04-11 07:28:12 -07:00
Awni Hannun
b8a348c1b8
Switch to fast RMS/LN Norm ( #603 )
...
* use nn.RMSNorm, use sdpa, cleanup
* bump mlx versions
* minor update
* use fast layer norm
* version bump
* update requirement for whisper
* update requirement for gguf
2024-03-23 07:13:51 -07:00
madroid
b0bcd86a40
Support for OpenAI’s fine-tuning dataset format ( #548 )
...
* LoRA: move load_dataset to tuner/datasets.py file
* LoRA: support OpenAI chat format datasets
see https://platform.openai.com/docs/guides/fine-tuning/example-format
* LoRA: support OpenAI completion format datasets
* LoRA: formatting dataset timing to reduce memory footprint
* Refactor dataset item access in PromptCompletionDataset
* Update mlx_lm/LORA.md
* Update mlx_lm/LORA.md
* check Unsupported data format
* add tests, fine-tune doc
* add tests, fine-tune doc
* add jinja2 for chat template
* nits in readme
* nits in readme
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2024-03-19 16:45:46 -07:00
Awni Hannun
8b05bb6d18
[mlx-lm] Use sdpa in llama / mistral model ( #515 )
...
* use sdpa
* update a few more models
* version
* fix stablelm type
2024-03-07 17:41:23 -08:00
Awni Hannun
f24edfa9dc
[mlx-lm] Add precompiled normalizations ( #451 )
...
* add precompiled normalizations
* nits
2024-02-22 12:40:55 -08:00
Awni Hannun
ab9172baac
Gemma support ( #474 )
...
* gemma support
* format
* lora support for gemma
2024-02-21 08:47:13 -08:00
Awni Hannun
8fd953ee2b
Support for slerp merging models ( #455 )
...
* support for slerp merging models
* docs
* update docs
* format'
2024-02-19 20:37:15 -08:00
Awni Hannun
f45a1ab83c
Update a few examples to use compile ( #420 )
...
* update a few examples to use compile
* update mnist
* add compile to vae and rename some stuff for simplicity
* update reqs
* use state in eval
* GCN example with RNG + dropout
* add a bit of prefetching
2024-02-08 13:00:41 -08:00
Anchen
30be4c4734
refactor(qwen): moving qwen into mlx-lm ( #312 )
...
* refactor(qwen): moving qwen into mlx-lm
* chore: update doc
* chore: fix type hint
* add qwen model support in convert
* chore: fix doc
* chore: only load model in quantize_model
* chore: make the convert script only copy tokenizer files instead of load it and save
* chore: update docstring
* chore: remove unnecessary try catch
* chore: clean up for tokenizer and update transformers 4.37
* nits in README
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2024-01-22 15:00:07 -08:00
Awni Hannun
c6440416a2
Mlx llm package ( #301 )
...
* fix converter
* add recursive files
* remove gitignore
* remove gitignore
* add packages properly
* read me update
* remove dup readme
* relative
* fix convert
* fix community name
* fix url
* version
2024-01-12 10:25:56 -08:00
