zhangyiss/gguf-tools
1
0
Fork 0
mirror of https://github.com/antirez/gguf-tools.git synced 2025-09-17 10:48:06 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

FP16 added. Split-mixtral improved.

Browse Source
This commit is contained in:
antirez
2023-12-27 15:13:42 +01:00
parent a77a4d061c
commit bd4ecbda94
6 changed files with 144 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
README.md
View File

@@ -1,6 +1,18 @@
# GGUF tools
Todo... still not clear what this is going to be.
This is a work in progress library to manipulate GGUF files.
The program 'gguf-tools' use the library to implement both useful and
useless stuff, to show the library usage.
gguf-tools show file.gguf
shows detailed info about the GGUF file. This will include all the key-value pairs, including arrays, and detailed tensors informations. Tensor offsets will be relative to the start *of the file*, not the start of the data section like in the GGUF format (absolute file offsets are more useful and simpler to use).
gguf-tools split-mixtral 65230776370407150546470161412165 mixtral.gguf out.gguf
Extracts a 7B model `out.gguf` from Mixtral 7B MoE using the specified MoE ID for each layer (there are 32 digits in the sequence 652...).
Note that split-mixtral is quite useless as models obtained in this way will not perform any useful action. This is just an experiment and a non trivial task to show how to use the library. Likely it will be removed soon, once I have more interesting and useful examples to show.
## Specification documents