4 Commits

Author	SHA1	Message	Date
Awni Hannun	fca087be49	More cache improvements (#1015) ... * fix rotating kv cache for chat use case * reorg + fixes to caching, unify prompt caching across types and use cases for e.g. caching during a chat * nit in chat * fix tests * fix tests * fix tests * docs * chat command * comments + docs * Define meta_state on all Cache implementations * fixes + trim_prompt_cache api * fix default model --------- Co-authored-by: Angelos Katharopoulos <a_katharopoulos@apple.com>	2024-10-07 20:45:51 -07:00
Alvaro Bartolome	f4709cb807	Align CLI args and some smaller fixes (#167) ... * Add `.DS_Store` files to `.gitignore` * Fix variable naming of `config` in `mixtral/convert.py` * Align CLI args and minor fixes * standardize * one more --------- Co-authored-by: Awni Hannun <awni@apple.com>	2023-12-22 14:34:32 -08:00
adhishthite	9cf82a0d43	Benchmark all models if user allows.	2023-12-07 00:07:42 +05:30
Awni Hannun	b243c1d8f4	a few examples	2023-11-29 08:17:26 -08:00