zhangyiss/gguf-tools
1
0
Fork 0
mirror of https://github.com/antirez/gguf-tools.git synced 2025-12-16 00:18:52 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Show subcommand: print total parameters.

Browse Source
This commit is contained in:
antirez
2023-12-28 16:06:52 +01:00
parent e2062eea2c
commit 2a599dc5d0
2 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -6,13 +6,13 @@ useless stuff, to show the library usage.
gguf-tools show file.gguf
shows detailed info about the GGUF file. This will include all the key-value pairs, including arrays, and detailed tensors informations. Tensor offsets will be relative to the start *of the file*, not the start of the data section like in the GGUF format (absolute file offsets are more useful and simpler to use).
shows detailed info about the GGUF file. This will include all the key-value pairs, including arrays, and detailed tensors informations. Tensor offsets will be relative to the start *of the file* (so they are actually absolute offsets), not the start of the data section like in the GGUF format.
gguf-tools split-mixtral 65230776370407150546470161412165 mixtral.gguf out.gguf
Extracts a 7B model `out.gguf` from Mixtral 7B MoE using the specified MoE ID for each layer (there are 32 digits in the sequence 652...).
Note that split-mixtral is quite useless as models obtained in this way will not perform any useful action. This is just an experiment and a non trivial task to show how to use the library. Likely it will be removed soon, once I have more interesting and useful examples to show.
Note that split-mixtral is quite useless as models obtained in this way will not perform any useful action. This is just an experiment and a non trivial task to show how to use the library. Likely it will be removed soon, once I have more interesting and useful examples to show, like models merging.
## Specification documents

4
gguf-tools.c
View File

@@ -163,6 +163,7 @@ void gguf_tools_show(const char *filename) {
/* Show all the tensors. */
gguf_tensor tensor;
uint64_t params = 0;
while (gguf_get_tensor(ctx,&tensor)) {
printf("%s tensor %.*s @%llu, %llu weights, %llu bytes\n",
gguf_get_tensor_type_name(tensor.type),
@@ -171,7 +172,10 @@ void gguf_tools_show(const char *filename) {
tensor.offset,
tensor.num_weights,
tensor.bsize);
params += tensor.num_weights;
}
printf("gguf-tools.info.parameters: %.02fB\n",
(double)params/1000000000);
return;
}