mirror of
https://github.com/ml-explore/mlx-examples.git
synced 2025-09-01 04:14:38 +08:00
Add dropout parameter to lora configuration (#599)
* Add dropout parameter to lora configuration A dropout parameter has been added to the lora configuration settings in lora_config.yaml. The LoRALinear class in utils.py has been updated to take this new parameter. Additionally, a AttributeError: 'types.SimpleNamespace' object has no attribute 'prompt' related to `args.prompt` has been removed from lora.py. * Update lora_config.yaml Set dropout to 0.0 in the sample config file * format --------- Co-authored-by: Awni Hannun <awni@apple.com>
This commit is contained in:
@@ -32,7 +32,11 @@ def linear_to_lora_layers(
|
||||
)
|
||||
|
||||
to_lora = lambda lin: LoRALinear.from_linear(
|
||||
lin, r=config["rank"], alpha=config["alpha"], scale=config["scale"]
|
||||
lin,
|
||||
r=config["rank"],
|
||||
alpha=config["alpha"],
|
||||
scale=config["scale"],
|
||||
dropout=config["dropout"],
|
||||
)
|
||||
|
||||
keys = config.get("keys", None)
|
||||
|
Reference in New Issue
Block a user