Alex Barron 85ffd2c96a Quantized KV Cache (#1075) ...

* add QuantizedKVCache

* simplify

* add tests

* single sdpa function

* fix sed

* in place

* fix tests

* support different k and v head dims

2024-10-31 16:59:52 -07:00

7.8 KiB

Raw Blame History

View Raw