Prevent llms/mlx_lm from serving the local directory as a webserver (#498)

* Don't serve local directory

BaseHTTPRequestHandler serves the current directory by default. Definitely not intended behaviour. Remove the "do_HEAD" and "do_GET" methods.

* Fix typo in method name

I assume hanlde_stream was intended to be called handle_stream

* Fix outdated typehint

load_model returns nn.Module, however fetch_from_hub was not updated to reflect the change

* Add some more type hints

* Add warnings for using in prod

Add a warning to README and runtime, discouraging use in production. The warning is the same as on the python docs for HTTPServer https://docs.python.org/3/library/http.server.html

* format

* nits

---------

Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
Y4hL
2024-02-28 05:40:42 +02:00
committed by GitHub
parent 676e574eff
commit ea92f623d6
5 changed files with 32 additions and 9 deletions

View File

@@ -7,6 +7,7 @@ import shutil
from pathlib import Path
import mlx.core as mx
import mlx.nn as nn
import numpy as np
import yaml
from mlx.utils import tree_flatten, tree_map
@@ -68,7 +69,7 @@ def slerp(t, w1, w2, eps=1e-5):
return s1 * w1 + s2 * w2
def merge_models(base_model, model, config):
def merge_models(base_model: nn.Module, model: nn.Module, config: dict):
method = config.get("method", None)
if method != "slerp":
raise ValueError(f"Merge method {method} not supported")