Awni Hannun ee60e2a9d5 Kv cache (#643) ...

* in place kv_cache

* fix

* fix kv cache size

* partially fix kv cache dtype

* step kv cache

* multiple of step size

* more teests + kv cache

* more kv cache

* udpate all models to use kv cache

2024-05-08 08:18:13 -07:00

..

__init__.py

Mlx llm package (#301)

2024-01-12 10:25:56 -08:00

base.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

cohere.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

dbrx.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

gemma.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

llama.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

minicpm.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

mixtral.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

olmo.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

openelm.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

phi3.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

phi.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

phixtral.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

plamo.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

qwen2_moe.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

qwen2.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

qwen.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

stablelm.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00

starcoder2.py

Kv cache (#643)

2024-05-08 08:18:13 -07:00