mark e5c98f4715 Update utils.py ...

Change to enable saving the kv-cache as a safetensors file after a text completion; after generate step has finished creating all the tokens, the key values cache is made into a dict and saved using mx.save_safetensors to a user-specified file location; similar to cache_prompt.

2024-09-26 16:58:02 +01:00

26 KiB

Raw Blame History

View Raw