From ee2ee0f8e55effcf341ae841cae0ba4abbdb63e9 Mon Sep 17 00:00:00 2001
From: Awni Hannun <awni@apple.com>
Date: Fri, 15 Dec 2023 12:18:29 -0800
Subject: [PATCH] 32 GB example

---
 lora/README.md | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/lora/README.md b/lora/README.md
index 2be2d78d..0b1b699c 100644
--- a/lora/README.md
+++ b/lora/README.md
@@ -150,6 +150,19 @@ of memory. Here are some tips to reduce memory use should you need to do so:
    you can do is break your examples into smaller
    sequences when making the `{train, valid, test}.jsonl` files.
 
+For example, for a machine with 32 GB the following should run reasonably fast:
+
+```
+python lora.py \
+   --model <path_to_model> \
+   --train \
+   --batch-size 1 \
+   --lora-layers 4
+```
+
+On an M1 Max with 32 GB we process about 250 tokens-per-second.
+
+
 [^lora]: Refer to the [arXiv paper](https://arxiv.org/abs/2106.09685) for more details on LoRA.
 [^llama]: Refer to the [arXiv paper](https://arxiv.org/abs/2302.13971) and [blog post](https://ai.meta.com/blog/large-language-model-llama-meta-ai/) for more details.
 [^mistral]: Refer to the [blog post](https://mistral.ai/news/announcing-mistral-7b/) and [github repository](https://github.com/mistralai/mistral-src) for more details.