Cheng
e776c970f7
Fix llava model when using text-only prompt ( #998 )
2024-09-25 07:19:41 -07:00
Albert Avetisian
bfbc0e434a
Add optional EOS token for llava example ( #753 )
...
* add optional EOS token
* add tokenizer config to align with MLX LM example
* formtatting fixes
2024-05-08 06:04:36 -07:00
Awni Hannun
b8a348c1b8
Switch to fast RMS/LN Norm ( #603 )
...
* use nn.RMSNorm, use sdpa, cleanup
* bump mlx versions
* minor update
* use fast layer norm
* version bump
* update requirement for whisper
* update requirement for gguf
2024-03-23 07:13:51 -07:00
Noah Kasmanoff
a429263905
LlaVA in MLX ( #461 )
...
* add: llava mlx first draft
* add: weights comparision
* add forward pass skeleton
* update: now imports weights correctly
* delete base
* latest
* adding config
* fix: use config
* add mlx config
* feat: add image processor for llava processor
* wip
* feat: llava working example
* chore: refactor generate script
* chore: clean up
* add: warning to user if no <image> token despite using one
* add: __call__ to LlavaModel
* add: call to LlavaModel
* update fp
* clean up var names
* update: native GeLU
* Cleanup
* update generate and readme
* remove todo comment
* rearrange tests
* fix example code
* nits in README
* update readme
* nit in readme
* nits in README
* chore(llava): refactor image embedding merging logic
* min mlx version
* nits in readmes
* fix cli prompt, some nits
* updates, slight simplify
---------
Co-authored-by: anchen <li.anchen.au@gmail.com >
Co-authored-by: Awni Hannun <awni@apple.com >
2024-03-01 10:28:35 -08:00
