4 Commits

Author	SHA1	Message	Date
Awni Hannun	6b209c6d3e	fix eos handling in stream generate	2024-11-06 06:48:05 -08:00
Anchen	82e3338987	chore(mlx-lm): add max token arg for mlx_lm.chat (#1089) ... * chore(mlx-lm): add max token arg for mlx_lm.chat * chore: update the default max token value	2024-11-04 06:06:34 -08:00
Awni Hannun	9f34fdbda4	Wire models in MLX LM (#1069) ... * wired in MLX LM * fix synch * comment + nit * version * mlx lm version * bump to 0.19.2	2024-10-31 08:17:14 -07:00
Awni Hannun	fca087be49	More cache improvements (#1015) ... * fix rotating kv cache for chat use case * reorg + fixes to caching, unify prompt caching across types and use cases for e.g. caching during a chat * nit in chat * fix tests * fix tests * fix tests * docs * chat command * comments + docs * Define meta_state on all Cache implementations * fixes + trim_prompt_cache api * fix default model --------- Co-authored-by: Angelos Katharopoulos <a_katharopoulos@apple.com>	2024-10-07 20:45:51 -07:00