Prince Canuma 76c3244cc5 Add support for Cohere's Command-R (#565) ...

* initial commit for command-R

* update mlp, layernorm, lm_head and model args

* add custom layernorm

* add default to tie_word_embeddings

* add layernorm weight type and refactor

* update layernorm (bias conditional) in model/layers

* fix layer norm use traditional rope

* add test

---------

Co-authored-by: Awni Hannun <awni@apple.com>

2024-03-13 07:03:36 -07:00

..

test_lora.py

LoRA on all linear transformer block layers (#546)

2024-03-12 07:37:40 -07:00

test_models.py

Add support for Cohere's Command-R (#565)

2024-03-13 07:03:36 -07:00

test_utils.py

LoRA on all linear transformer block layers (#546)

2024-03-12 07:37:40 -07:00