James A Capozzoli 9717307ff0 Validation with full data set, results in NaN validation score (#879) ...

* CLI arguments may set num_batches to -1

The CLI arguments allow you to validate with the entire dataset by passing a negative one value, but this quickly results in a division by zero `NaN` to appear as the validation score!

* Must properly assemble the mini batches when validating with entire dataset.

Tested locally, a validation of a novel took about an hour, with a loss of 0.928. Thanks @awni for the correction!

* Set up the pre-commit hooks and run them so that black may format lora.py.

2024-07-10 08:36:11 -07:00

12 KiB

Raw Blame History

View Raw