zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-12-15 09:48:54 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Enable unit testing in Circle and start some MLX LM tests (#545)

Browse Source 
* add a few tests for mlx lm

* add a few tests for mlx lm

* add a few tests for mlx lm

* more tests / cleanup
This commit is contained in:
Awni Hannun
2024-03-07 09:31:57 -08:00
committed by GitHub
parent ef32379bc6
commit 7cdd1b69ac
12 changed files with 294 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
llms/mlx_lm/models/starcoder2.py
View File

@@ -15,10 +15,8 @@ class ModelArgs(BaseModelArgs):
num_hidden_layers: int
intermediate_size: int
num_attention_heads: int
num_key_value_heads: int = None
max_position_embeddings: int = 16384
num_key_value_heads: int
norm_epsilon: float = 1e-5
norm_type: str = "layer_norm"
vocab_size: int = 49152
rope_theta: float = 100000
tie_word_embeddings: bool = True