mlx_lm: Add Streaming Capability to Generate Function (#807)

* Add streaming feature to text generation function

* separate stream and regular functions

---------

Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
Michał Kurc
2024-06-03 18:04:39 +02:00
committed by GitHub
parent 8353bbbf93
commit 43d6deb3c1
4 changed files with 89 additions and 37 deletions

View File

@@ -149,10 +149,10 @@ def main():
model,
tokenizer,
prompt,
args.temp,
args.max_tokens,
True,
verbose=True,
formatter=formatter,
temp=args.temp,
top_p=args.top_p,
)