4 Commits

Author	SHA1	Message	Date
Gökdeniz Gülmez	deee214a95	Adding support for the Muon Optimizer (#1914) ... * initial commit with workong optmimizer * update ACKNOWLEDGMENTS.md * nits and adding it to test * nits * G.astype(mx.bfloat16) to G.astype(G.dtype) * G.ndim >= 2 to assert G.ndim == 2 * remove coments * replace with mx.addmm * remove comments * format * nits * match muon * fix addmm --------- Co-authored-by: Awni Hannun <awni@apple.com>	2025-07-18 12:25:28 -07:00
Awni Hannun	9c6953bda7	Fix stubgen (#2065) ... * Fix stubgen * add multi optim to docs	2025-04-11 12:02:54 -07:00
Chime Ogbuji	3b661b7394	Add linear warmup and schedule joining for use with existing schedules (#721) ... * Add linear warmup to schedules for use with existing schedules * Changed parameters for simplicity of most common case (0 initial value) * Added ScheduleJoiner and updated documentation * ScheduleJoiner -> join_schedules (ala optax #) * black compliance * Different evaluation of schedules * nits --------- Co-authored-by: Awni Hannun <awni@apple.com>	2024-02-26 07:28:48 -08:00
Srimukh Sripada	818cda16bc	Support LR schedulers (#334) ... * Add a few LR schedulers * Move parents's constructor call to the top * Fix docstring * refactor optimizers into two files * add docs * nit * Fix Callable type annotation for python 3.8 --------- Co-authored-by: Awni Hannun <awni@apple.com> Co-authored-by: Angelos Katharopoulos <a_katharopoulos@apple.com>	2024-02-15 11:26:20 -08:00