zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-09-01 12:49:50 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

add speculative decoding example for llama (#149)

Browse Source 
* speculative decoding

* add sample 0

* spec decode gives same results as regular decode

* rebase

* use accept reject criteria

* switch to t5

* update readme

* readme nit

* nits

* nits

* nits

---------

Co-authored-by: Benjamin Anderson <benjamin@Benjamins-MBP.lan>
Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
Benjamin Anderson
2023-12-28 17:20:43 -06:00
committed by GitHub
parent 07c163d9d9
commit 09566c7257
7 changed files with 775 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
llms/speculative_decoding/requirements.txt Normal file
View File

@@ -0,0 +1,3 @@
mlx>=0.0.6
transformers
numpy