Adds EXAONE architecture. (#1145)

* Adds EXAONE architecture. * nits + format * format * clean up and fix rope * clean up and fix rope --------- Co-authored-by: Awni Hannun <awni@apple.com>
2025-09-01 21:01:32 +08:00 · 2024-12-09 10:58:25 -05:00
parent 893b3f085e
commit 5687d5b99b
6 changed files with 312 additions and 224 deletions
--- a/llms/mlx_lm/tuner/utils.py
+++ b/llms/mlx_lm/tuner/utils.py
@@ -144,6 +144,8 @@ def linear_to_lora_layers(
                "mixer.out_proj",
            ]
        )
+    elif model.model_type == "exaone":
+        keys = set(["attn.attention.q_proj", "attn.attention.v_proj"])
    else:
        raise ValueError(f"Lora does not support {model.model_type}")