zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-09-01 12:49:50 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

[mlx-lm] Use top p in server (#1144)

Browse Source 
* use top p in server

* couple other fixes
This commit is contained in:
Awni Hannun
2024-12-12 11:12:21 -08:00
committed by GitHub
parent 19abf3dcaa
commit 2ba0e36683
3 changed files with 5 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
llms/mlx_lm/server.py
View File

@@ -465,7 +465,7 @@ class APIHandler(BaseHTTPRequestHandler):
text = ""
tic = time.perf_counter()
sampler = make_sampler(self.temperature)
sampler = make_sampler(self.temperature, top_p=self.top_p)
logits_processors = make_logits_processors(
self.logit_bias, self.repetition_penalty, self.repetition_context_size
)