Gökdeniz Gülmez 56d2db23e1 adding OLMoE architecture (#1321) ...

* initial commit

* udpate ACKNOWLEDGMENTS.md

* adding olmoe to training

* clean up

* faster generation

* remove sanitize method

* more clean ups

* adding SwitchGLU

* clean up

* a little faster and adding norm_topk_prob

* formated

2025-03-05 13:46:06 -08:00

7.3 KiB

Raw Blame History

View Raw