zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-06-25 01:41:19 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1e05aef344
mlx-examples/llms/mlx_lm/models
History
n8programs 1e05aef344
Add logit soft capping to gemma, and fix precision issues (#857) 
* Add logit soft capping to gemma, and fix precision issues

Gemma was babbling nonsense - so I figured out it was due to not having logit softcapping and precision issues causing NaNs (so I implemented the softcapping and added more float32 inference). gemma-27b-it-4bit now works flawlessly (or near-flawlessly, no sliding-window attention).

* get rid of comments

* get rid of last comments (sry lol)

* nits

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2024-07-02 07:52:39 -07:00
..
__init__.py Mlx llm package (#301) 2024-01-12 10:25:56 -08:00
base.py Add support for ibm granite (#758) 2024-05-21 20:16:31 -07:00
cohere.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
dbrx.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
gemma2.py Add logit soft capping to gemma, and fix precision issues (#857) 2024-07-02 07:52:39 -07:00
gemma.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
gpt2.py GPT2 Support (#798) 2024-06-02 16:33:20 -07:00
gpt_bigcode.py Add support for ibm granite (#758) 2024-05-21 20:16:31 -07:00
internlm2.py support internlm2 (#797) 2024-05-27 06:22:21 -07:00
llama.py Correct the type annotation of cache in llama.py (#828) 2024-06-10 15:18:34 -07:00
minicpm.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
mixtral.py fix moe conversion (#802) 2024-05-31 12:36:05 -07:00
olmo.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
openelm.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
phi3.py make models/phi3.py and models/phi3small.py compatible with mypy (#833) 2024-06-12 06:53:55 -07:00
phi3small.py make models/phi3.py and models/phi3small.py compatible with mypy (#833) 2024-06-12 06:53:55 -07:00
phi.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
phixtral.py fix moe conversion (#802) 2024-05-31 12:36:05 -07:00
plamo.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
qwen2_moe.py Fix mypy errors with models/{qwen2,qwen2_moe,startcoder2}.py (#835) 2024-06-14 09:44:50 -07:00
qwen2.py Fix mypy errors with models/{qwen2,qwen2_moe,startcoder2}.py (#835) 2024-06-14 09:44:50 -07:00
qwen.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
stablelm.py Kv cache (#643) 2024-05-08 08:18:13 -07:00
starcoder2.py Fix mypy errors with models/{qwen2,qwen2_moe,startcoder2}.py (#835) 2024-06-14 09:44:50 -07:00
su_rope.py Su-RoPE(Rotary Position Embedding) for Phi-3 (#813) 2024-06-11 06:20:04 -07:00
switch_layers.py Rename block sparse to gather (#793) 2024-05-23 19:47:35 -07:00