8 Commits

Author	SHA1	Message	Date
Awni Hannun	9f2ea5892e	Bpe stream without space (#1154) ... * bpe streaming detokenization without space * version bump	2024-12-12 13:13:50 -08:00
Awni Hannun	1963df8565	Allow prompt callback to generate_step (#1133) ... * allow prompt callback and use in cache_prompt * nit * comments * bump version	2024-12-03 16:17:14 -08:00
Awni Hannun	cfc29c29f4	Put prompt processing in same stream (#1122) ... * put prompt processing in same stream * patch	2024-11-25 09:47:00 -08:00
Awni Hannun	0f135396ae	Generation refactor: part 2 (#1099) ... * unify with stream_generate * fixes * nit * some cleanup, warnings, tests * fix test + faster min p + test * version	2024-11-23 11:47:06 -08:00
Awni Hannun	9f34fdbda4	Wire models in MLX LM (#1069) ... * wired in MLX LM * fix synch * comment + nit * version * mlx lm version * bump to 0.19.2	2024-10-31 08:17:14 -07:00
Awni Hannun	fca087be49	More cache improvements (#1015) ... * fix rotating kv cache for chat use case * reorg + fixes to caching, unify prompt caching across types and use cases for e.g. caching during a chat * nit in chat * fix tests * fix tests * fix tests * docs * chat command * comments + docs * Define meta_state on all Cache implementations * fixes + trim_prompt_cache api * fix default model --------- Co-authored-by: Angelos Katharopoulos <a_katharopoulos@apple.com>	2024-10-07 20:45:51 -07:00
Awni Hannun	c3e3411756	Update LLM generation docs to use chat template (#973) ... * fix docs * add template to model cards as well * revert * version	2024-09-07 06:06:15 -07:00
James Zhao	bf921afcbe	Make sure to import the correct "version" module when installing mlx_whisper and mlx_lm from local source code. (#969) ... * Make sure to import the correct "version" module when installing the mlx_whisper package from local source code. * Make sure to import the correct "version" module when installing the mlx_lm package from local source code * fix --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-09-03 13:16:21 -07:00