zhangyiss/mlx-examples
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx-examples.git synced 2025-08-29 18:26:37 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

nits

Browse Source
This commit is contained in:
Awni Hannun 2024-10-22 12:16:59 -07:00
parent b36625aa6d
commit 43d0401bcf
1 changed files with 11 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
llms/mlx_lm/LORA.md
View File

@ -223,11 +223,16 @@ data formats. Here are examples of these formats:
```
The format for defining the `arguments` field in a function can vary depending on the model being used. Common formats include JSON strings and dictionaries.
The example provided follows the format used by [OpenAI](https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples), which is also adopted by MistralAI [mistral-finetune](https://github.com/mistralai/mistral-finetune?tab=readme-ov-file#instruct).
However, in Hugging Face's [chat_template](https://huggingface.co/docs/transformers/main/en/chat_templating#a-complete-tool-use-example), a dictionary format is used.
The choice of format should depend on the base model you are fine-tuning. It is recommended to refer to the documentation of the base model for detailed instructions regarding this aspect.
The format for the `arguments` field in a function varies for different models.
Common formats include JSON strings and dictionaries. The example provided
follows the format used by
[OpenAI](https://platform.openai.com/docs/guides/fine-tuning/fine-tuning-examples),
and [Mistral
AI](https://github.com/mistralai/mistral-finetune?tab=readme-ov-file#instruct).
In Hugging Face's [chat
templates](https://huggingface.co/docs/transformers/main/en/chat_templating#a-complete-tool-use-example),
a dictionary format is used. Refer to the documentation for the model you are
fine-tuning for more details.
</details>
@ -248,7 +253,7 @@ each line not expected by the loader will be ignored.
> [!NOTE]
> Each example in the datasets must be on a single line. Do not put more than
> one example per line and do not split an example accross multiple lines.
> one example per line and do not split an example across multiple lines.
### Hugging Face Datasets