Muhtasham Oblokulov 81e2a80026 Add Starcoder 2 (#502) ... * Add Starcoder2 model and update utils.py * Refactor model arguments and modules in starcoder2.py * Refactor FeedForward class to MLP in starcoder2.py * Fix typo * pre-commit * Refactor starcoder2.py: Update model arguments and modules * Fix LM head and MLP layers * Rename input layer norm * Update bias in linear layers * Refactor token embeddings in Starcoder2Model * Rename to standard HF attention layer name * Add LayerNorm * Add transposed token embeddings (like in Gemma) * Refactor MLP and TransformerBlock classes * Add tie_word_embeddings option to ModelArgs and update Model implementation * Add conditional check for tying word embeddings in Starcoder2Model * Fix bias in lm_head linear layer * Remove unused LayerNorm in stablelm * Update transformers dependency to use GitHub repository * fix lm head bug, revert transformer req * Update RoPE initialization in Attention class --------- Co-authored-by: Awni Hannun <awni@apple.com>		2024-03-02 19:39:23 -08:00
..
__init__.py	feat: move lora into mlx-lm (#337)	2024-01-23 08:44:37 -08:00
lora.py	feat(mlx-lm): add de-quant for fuse.py (#365)	2024-01-25 18:59:32 -08:00
trainer.py	LoRA: adapter file Support path information (#505)	2024-02-29 22:20:49 -08:00
utils.py	Add Starcoder 2 (#502)	2024-03-02 19:39:23 -08:00