zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-09-01 04:14:38 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

LlaVA in MLX (#461)

Browse Source 
* add: llava mlx first draft

* add: weights comparision

* add forward pass skeleton

* update: now  imports weights correctly

* delete base

* latest

* adding config

* fix: use config

* add mlx config

* feat: add image processor for llava processor

* wip

* feat: llava working example

* chore: refactor generate script

* chore: clean up

* add: warning to user if no <image> token despite using one

* add: __call__ to LlavaModel

* add: call to LlavaModel

* update fp

* clean up var names

* update: native GeLU

* Cleanup

* update generate and readme

* remove todo comment

* rearrange tests

* fix example code

* nits in README

* update readme

* nit in readme

* nits in README

* chore(llava): refactor image embedding merging logic

* min mlx version

* nits in readmes

* fix cli prompt, some nits

* updates, slight simplify

---------

Co-authored-by: anchen <li.anchen.au@gmail.com>
Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
Noah Kasmanoff
2024-03-01 13:28:35 -05:00
committed by GitHub
parent 261f1280f6
commit a429263905
9 changed files with 994 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
llava/requirements.txt Normal file
View File

@@ -0,0 +1,6 @@
mlx>=0.5.0
numpy
transformers
torch
huggingface_hub
Pillow