zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-12-16 02:08:55 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
Files
424cb854e930ef0bced08e260a5eeea28bd566bc
mlx-examples/llms/tests/test_tokenizers.py
Billel Mokeddem 845efddc8c Fix decoding manually added tokens (#1164) 
* Fix decoding manually added tokens

* fix + test

* nit

* nit

* no lag bpe

---------

Co-authored-by: Awni Hannun <awni@apple.com>
2024-12-17 09:54:29 -08:00

3.4 KiB
Raw Blame History

View Raw
Reference in New Issue View Git Blame Copy Permalink