Nripesh Niketan
e09bf35b28
feat: Add Dropout3d layer to nn.layers ( #313 )
...
* feat: Add Dropout3d layer to nn.layers
* acknowledgement
* Add dropout tests to test_nn.py
* run pre-commit
* Add activation functions and dropout3d ops
* Add dropout tests for bfloat16 and float16
2023-12-31 14:01:21 -08:00
Hazem Essam
e3b8da2a49
Added implementation for Scaled RoPE. ( #261 )
...
* Added scale for RoPE
* Ran pre-commit
* Added RoPE scaling test
* Added docstring for scale parameter
* Modified docstrings
2023-12-31 06:06:01 -08:00
Nripesh Niketan
5ad8fb7268
feat: add softsign, softmax, hardswish, logsoftmax activation function ( #309 )
...
* feat: add softsign activation function
* run pre-commit
* Add Softsign activation function
* Add Softsign activation function
* Add documentation for ReLU6, Softplus, and Softsign activations
* Update activation functions in neural network layers
* Add LogSoftmax and Hardswish activations
* run pre-commit
* Update activations.py
* Added acknowledgements
* Fix activation function comments
* Fix activation functions in neural network layers
2023-12-29 11:49:36 -08:00
Angelos Katharopoulos
d29770eeaa
Update batchnorm to have the running stats in parameters ( #305 )
2023-12-28 14:31:10 -08:00
__mo_san__
a123c3c7d2
implement-batch-norm-layer ( #217 )
...
- Add batch normalization layer
---------
Co-authored-by: Robert McCraith <mccraithrobert@gmail.com>
Co-authored-by: Awni Hannun <awni@apple.com>
2023-12-25 07:32:53 -08:00
Vidit Agarwal
8c3da54c7d
Fix failing test for log cosh loss ( #275 )
...
* fix assert statement in log_cosh_loss
* reformatted by pre-commit black
2023-12-23 16:26:46 -08:00
Nicholas Santavas
d35fa1db41
Add Hinge, Huber and LogCosh losses ( #199 )
2023-12-22 10:28:10 -08:00
Angelos Katharopoulos
1d053e0d1d
Fix the alibi test that was left unchanged ( #252 )
2023-12-21 14:59:25 -08:00
Hazem Essam
0aa65c7a6b
Added ALiBi implementation ( #232 )
2023-12-21 14:36:38 -08:00
Emircan Erol
e549f84532
Triplet Loss ( #211 )
...
* Triplet Loss
* Requested Changes
* Margin to alpha
2023-12-19 12:37:12 -08:00
jojopuppet
18cca64c81
Add smoothed L1 loss and enhancements to cross entropy loss ( #166 )
...
* Add smooth_l1_loss
* Add labels moothing for cross entropy loss
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2023-12-18 07:26:21 -08:00
Diogo
dc2edc762c
added tri / tril / triu ( #170 )
...
* added tri / tril / triu
* fixed tests
* ctest tests
* tri overload and simplified tests
* changes from comment
* more tests for m
* ensure assert if not 2-D
* remove broadcast_to
* minor tweaks
---------
Co-authored-by: Awni Hannun <awni@apple.com>
2023-12-15 17:30:34 -08:00
Awni Hannun
2e02acdc83
add base kwarg to rope ( #186 )
2023-12-15 16:47:59 -08:00
Diogo
02de234ef0
Activations LeakyReLU / PReLU / Softplus / Mish ( #109 )
...
* Leaky_relu / prelu / softplus / mish
* added tests
* updated bench
* remove torch refs, add init to PReLU
* added arvix reference to mish
* added missing docs
2023-12-11 19:40:57 -08:00
Nicholas Santavas
f5df47ec6e
Add Step, ELU, SELU, Swish activation functions ( #117 )
...
* Add Step, ELU, SELU, Swish activation functions
This commit adds the Step, ELU, SELU and Swish activations functions
* add to the docs
* review
2023-12-11 17:04:07 -08:00
__mo_san__
072044e28f
fix and update binary cross entropy loss tests ( #133 )
...
* fix conflicts
* updated tests
2023-12-11 12:42:17 -08:00
Awni Hannun
69505b4e9b
fixes ( #131 )
2023-12-11 09:26:49 -08:00
__mo_san__
f4ddd7dc44
Add Binary Cross Entropy loss ( #122 )
...
* update BCE added tests for it ...
* added binary cross entropy loss to docs
* resolving conflicts for merge
2023-12-11 07:55:18 -08:00
Jason
b0cd092b7f
Added activation functions: leaky_relu relu6 softplus elu celu logsigmoid ( #108 )
...
* added leaky_relu relu6 softplus elu celu logsigmoid
* minor fixes for docstring and benchmark imports
* fixed elu implementation and added tests
* added tests for optional param, changed leaky_relu param to fit pytorch documentation
2023-12-10 16:31:38 -08:00
Awni Hannun
2d0130f80f
fix loss tests ( #118 )
...
* fix loss tests
* use none as default
2023-12-10 10:08:19 -08:00
Angelos Katharopoulos
600db7d754
Fix build on Xcode 14 ( #116 )
...
* Fix build on Xcode 14
* Style fixes
2023-12-10 06:58:52 -08:00
Enoch Kan
0b28399638
added mse_loss, nll_loss and kl_div_loss ( #98 )
...
* added mse_loss, nll_loss and kl_div_loss
* fixed axis not defined error in nll_loss
* fixed axis not defined in kl_div_loss
* added tests for mse, nll and kl_div
* modified docstrings and added reduce helper func
* updated docstring in kl_div_loss and moved helper func
* added new kl divergence implementation
* added reduction to test
* updated docstring of kl_div_loss with correct spelling
* added losses to nn.rst in docs
2023-12-09 14:25:03 -08:00
Kai Ma
cb9e585b8e
Style fix for loss functions ( #91 )
...
* MLE and L1 loss functions
* logsoftmax change and tests
* subtract max logit for numerical stability
* l1 name change
* cross entropy reduction + unit tests
* docstrings
* l1 test name change
* old loss impl + default none
* style
2023-12-08 21:11:56 -08:00
Kai Ma
641d316484
MLE and L1 loss functions ( #88 )
...
* MLE and L1 loss functions
* logsoftmax change and tests
* subtract max logit for numerical stability
* l1 name change
* cross entropy reduction + unit tests
* docstrings
* l1 test name change
* old loss impl + default none
2023-12-08 20:21:37 -08:00
Zach Schillaci
5b9be57ac3
Add isort pre-commit and run ( #68 )
2023-12-08 11:31:47 -08:00
Awni Hannun
46a39e5b1f
copyright + ack
2023-11-30 11:12:53 -08:00
Jagrit Digani
e6306cfee9
jagrit's commit files
2023-11-29 10:52:08 -08:00
