21 Commits

Author	SHA1	Message	Date
Goekdeniz-Guelmez	3a499f9735	fixed inference slowness but it cant handle multible Token inputs and is generateing gibberish	2024-11-10 16:35:07 +01:00
Goekdeniz-Guelmez	800b60239c	save checkpoint	2024-11-10 14:36:26 +01:00
Goekdeniz-Guelmez	906f972d36	save push	2024-11-06 16:35:46 +01:00
Goekdeniz-Guelmez	58b448dc0b	updates	2024-10-30 21:23:13 +01:00
Goekdeniz-Guelmez	7c8849e795	update	2024-10-24 16:16:42 +02:00
Goekdeniz-Guelmez	a677638c4b	inference works but is hella slow	2024-10-22 23:06:06 +02:00
Goekdeniz-Guelmez	9ab581d678	notes	2024-10-22 22:10:53 +02:00
Goekdeniz-Guelmez	e43a2ab229	not working, incorrect handling with cache probably	2024-10-22 22:04:25 +02:00
Goekdeniz-Guelmez	55485b98e8	update	2024-10-22 21:23:47 +02:00
Goekdeniz-Guelmez	758597eaa8	adding multi token input and correct cache handling in ssm step	2024-10-22 20:44:23 +02:00
Goekdeniz-Guelmez	b9c57cd429	generation works! trying training now	2024-10-22 18:25:59 +02:00
Goekdeniz-Guelmez	c1634ce81b	still generating gibberish	2024-10-20 18:41:28 +02:00
Goekdeniz-Guelmez	ab4cf1d1cf	generation works but outputs gibberish	2024-10-20 18:04:34 +02:00
Goekdeniz-Guelmez	4ab5139c05	quick save	2024-10-20 16:11:39 +02:00
Goekdeniz-Guelmez	cd036ccfb5	fix generation works too (almost)	2024-10-16 21:13:36 +02:00
Goekdeniz-Guelmez	8073cb486c	adding debug statements (somehiw generating only goes through the fist MambaMixer block pass)	2024-10-16 21:09:30 +02:00
Goekdeniz-Guelmez	00ba27fe6c	adding debug statements	2024-10-11 21:36:41 +02:00
Goekdeniz-Guelmez	6f88dd59d7	quick clean up and fix	2024-10-11 21:08:13 +02:00
Goekdeniz-Guelmez	4e1236cbf6	fixing loading the model	2024-10-11 20:53:29 +02:00
Goekdeniz-Guelmez	264ba43707	update trainer/lora.py and adding DepthWiseConv1d because mlx 0.18.0 doesnt axepts groups parameter	2024-10-02 19:19:32 +02:00
Gökdeniz Gülmez	49b9fc1a4c	Create mamba2.py	2024-10-02 12:48:15 +02:00