5 Commits

Author	SHA1	Message	Date
Alex Barron	5b414dddf2	comment + fix	2024-12-18 18:39:14 -08:00
Alex Barron	cd9dcf0383	add mask to mlx_lm model interface	2024-12-18 13:54:14 -08:00
Alex Barron	85ffd2c96a	Quantized KV Cache (#1075) ... * add QuantizedKVCache * simplify * add tests * single sdpa function * fix sed * in place * fix tests * support different k and v head dims	2024-10-31 16:59:52 -07:00
Awni Hannun	fca087be49	More cache improvements (#1015) ... * fix rotating kv cache for chat use case * reorg + fixes to caching, unify prompt caching across types and use cases for e.g. caching during a chat * nit in chat * fix tests * fix tests * fix tests * docs * chat command * comments + docs * Define meta_state on all Cache implementations * fixes + trim_prompt_cache api * fix default model --------- Co-authored-by: Angelos Katharopoulos <a_katharopoulos@apple.com>	2024-10-07 20:45:51 -07:00
Prince Canuma	b5e18ef1e3	Add Phi-3.5-MoE (#946) ... * add phimoe * add phimoe to tunner * add switch_mlp * fix SuScaled args * nits --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-08-24 06:52:33 -07:00