zhangyiss/mlx
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx.git synced 2025-06-25 01:41:17 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
141 Commits 36 Branches 64 Tags 247 MiB
e09bf35b28
Commit Graph

141 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Nripesh Niketan
e09bf35b28
feat: Add Dropout3d layer to nn.layers (#313) 
* feat: Add Dropout3d layer to nn.layers

* acknowledgement

* Add dropout tests to test_nn.py

* run pre-commit

* Add activation functions and dropout3d ops

* Add dropout tests for bfloat16 and float16
 2023-12-31 14:01:21 -08:00
Daniel Strobusch
99c20f523e
fix typos (#327) 2023-12-31 06:06:47 -08:00
Hazem Essam
e3b8da2a49
Added implementation for Scaled RoPE. (#261) 
* Added scale for RoPE

* Ran pre-commit

* Added RoPE scaling test

* Added docstring for scale parameter

* Modified docstrings
 2023-12-31 06:06:01 -08:00
Angelos Katharopoulos
a020a2d49d
Improve repeat using broadcasting and reshape (#318) 2023-12-29 21:40:20 -08:00
Nripesh Niketan
930b159885
Fix: typo in README.md (#316) 2023-12-29 12:58:00 -08:00
Nripesh Niketan
5ad8fb7268
feat: add softsign, softmax, hardswish, logsoftmax activation function (#309) 
* feat: add softsign activation function

* run pre-commit

* Add Softsign activation function

* Add Softsign activation function

* Add documentation for ReLU6, Softplus, and Softsign activations

* Update activation functions in neural network layers

* Add LogSoftmax and Hardswish activations

* run pre-commit

* Update activations.py

* Added acknowledgements

* Fix activation function comments

* Fix activation functions in neural network layers
 2023-12-29 11:49:36 -08:00
Chunyang Wen
2aedf3e791
Minor refactor for tree_map and tree_unflatten (#311) 
* Minor refact for tree_map and tree_unflatten

* Remove the if statement

---------

Co-authored-by: Chunyang Wen <chunyang_wen@apple.com>
 2023-12-28 20:55:10 -08:00
Chunyang Wen
473b6b43b4
Use defaultdict (#307) 
Co-authored-by: Chunyang Wen <chunyang_wen@apple.com>
 2023-12-28 14:46:13 -08:00
Angelos Katharopoulos
d29770eeaa
Update batchnorm to have the running stats in parameters (#305) 2023-12-28 14:31:10 -08:00
Chunyang Wen
040c3bafab
Add missing f str (#306) 
Co-authored-by: Chunyang Wen <chunyang_wen@apple.com>
 2023-12-28 06:09:34 -08:00
Chunyang Wen
05767b026f
Add information for dropout probability (#304) 
Co-authored-by: Chunyang Wen <chunyang_wen@apple.com>
 2023-12-27 21:51:30 -08:00
Diogo
a83d5d60bd
Addition in acknowledgements (#302) 2023-12-27 13:46:47 -08:00
Bahaa
ff2b58e299
Add support for repeat (#278) 
* add repeat function

* fix styling

* optimizing repeat

* fixed minor issues

* not sure why that folder is there xD

* fixed now for sure

* test repeat not repeat test

* Fixed

---------

Co-authored-by: Bahaa Eddin tabbakha <bahaa@Bahaas-MacBook-Pro.local>
 2023-12-27 13:11:38 -08:00
YUN, Junwoo
4417e37ede
Transformer fix (#167) 
* add transformer with dropout, fix transformer ffm, layernorm order

* precommit changes

* precommit changes

* add docstring, activation, norm_first

* run precommit

* run precommit

* add doctstring

* precommit

* style nits in docs

---------

Co-authored-by: junwoo-yun <junwoo.yun@bagelcode.com>
Co-authored-by: Awni Hannun <awni@apple.com>
 2023-12-27 08:48:36 -08:00
Angelos Katharopoulos
79c95b6919
Fix load compilation (#298) 2023-12-27 06:20:45 -08:00
Diogo
1f6ab6a556
Safetensor support (#215) 
Co-authored-by: Awni Hannun <awni@apple.com>
 2023-12-27 02:06:55 -08:00
Gabrijel Boduljak
6b0d30bb85
linalg.norm (#187) 
* implemented vector_norm in cpp

added linalg to mlx

* implemented vector_norm python binding

* renamed vector_norm to norm, implemented norm without provided ord

* completed the implementation of the norm

* added tests

* removed unused import in linalg.cpp

* updated python bindings

* added some tests for python bindings

* handling inf, -inf as numpy does, more extensive tests of compatibility with numpy

* added better docs and examples

* refactored mlx.linalg.norm bindings

* reused existing util for implementation of linalg.norm

* more tests

* fixed a bug with no ord and axis provided

* removed unused imports

* some style and API consistency updates to linalg norm

* remove unused includes

* fix python tests

* fixed a bug with frobenius norm of a complex-valued matrix

* complex for vector too

---------

Co-authored-by: Awni Hannun <awni@apple.com>
 2023-12-26 19:42:04 -08:00
Angelos Katharopoulos
447bc089b9
Fix tolerance in de-/quantization test (#295) 2023-12-26 19:21:05 -08:00
Yutaka Kondo
fc4e5b476b
Fix llama link in README.md (#289) 2023-12-25 20:53:20 -08:00
Daniel Strobusch
d58ac083f3
expose itemsize and nbytes as for numpy arrays (#284) 
see:
  * https://numpy.org/doc/stable/reference/generated/numpy.ndarray.nbytes.html
  * https://numpy.org/doc/stable/reference/generated/numpy.ndarray.itemsize.html

relates to https://github.com/ml-explore/mlx-examples/pull/174
 2023-12-25 10:34:28 -08:00
__mo_san__
a123c3c7d2
implement-batch-norm-layer (#217) 
- Add batch normalization layer

---------

Co-authored-by: Robert McCraith <mccraithrobert@gmail.com>
Co-authored-by: Awni Hannun <awni@apple.com>
 2023-12-25 07:32:53 -08:00
Angelos Katharopoulos
9e6b8c9f48
Refactor the reduction kernels (#277) 2023-12-24 14:47:57 -08:00
Zach Schillaci
22fee5a383
Remove redundant assert in losses.py (#281) 2023-12-24 08:39:08 -08:00
Daniel Strobusch
7365d142a3
random.uniform must respect dtype, even if lower precision than "low" (#280) 
Fix an edge case where random uniform returns a float32 array, even if a lower precision dtype is wanted due to adding the float32 "low" array.
 2023-12-24 07:04:43 -08:00
Awni Hannun
8b227fa9af
fix no metal build (#276) 2023-12-23 19:18:10 -08:00
Vidit Agarwal
8c3da54c7d
Fix failing test for log cosh loss (#275) 
* fix assert statement in log_cosh_loss

* reformatted by pre-commit black
 2023-12-23 16:26:46 -08:00
Vidit Agarwal
acf1721b98
Corrected the example of value_and_grad (#274) 
* Corrected the example for mx.value_and_grad

* Reformat through pre-commit/black
 2023-12-23 11:06:38 -08:00
Finn Voorhees
f91f450141
Fix argmax returns documentation (#263) 2023-12-22 20:33:17 -08:00
Ronan Collobert
cd3616a463
Revisit autorelease memory pools (#260) 
* make general autorelease pool part of metal device

* make things simpler

* no metal backend support

* new_memory_pool -> new_scoped_memory_pool
 2023-12-22 11:01:26 -08:00
Nicholas Santavas
d35fa1db41
Add Hinge, Huber and LogCosh losses (#199) 2023-12-22 10:28:10 -08:00
Justin Deschenaux
e8deca84e0
Add dropout2d (#250) 2023-12-22 08:02:29 -08:00
Angelos Katharopoulos
8385f93cea
Bumping the version (#256) 2023-12-21 18:33:14 -08:00
Awni Hannun
2118c3dbfa
fix (#255) 2023-12-21 18:18:41 -08:00
Awni Hannun
a002797d52
A temporary fix (#254) 2023-12-21 17:59:15 -08:00
Angelos Katharopoulos
1d053e0d1d
Fix the alibi test that was left unchanged (#252) 2023-12-21 14:59:25 -08:00
Hazem Essam
0aa65c7a6b
Added ALiBi implementation (#232) 2023-12-21 14:36:38 -08:00
Daniel Strobusch
794feb83df
support arange for bfloat16 (#245) 2023-12-21 14:33:43 -08:00
Angelos Katharopoulos
2c7df6795e
Make sure that arrays are freed when saving (#247) 2023-12-21 14:08:24 -08:00
Angelos Katharopoulos
b3916cbf2b
Improve names of quantization arguments (#235) 
* Change the default quantization group_size to 64
* Rename groups to group_size and width to bits
 2023-12-20 16:53:53 -08:00
Angelos Katharopoulos
57fe918cf8
Adds C++ and nn quantization utilities (#230) 
* Add C++ de-/quantize ops
* Add quantize functions to the docs and tests
* Add a QuantizedLinear module
 2023-12-20 14:17:38 -08:00
Justin Deschenaux
4912ff3ec2
Add Lion optimizer (#209) 
* Add Lion optimizer
* Update acknowledgements also with past contributions
 2023-12-20 13:54:58 -08:00
Awni Hannun
f40d17047d
Indexing bug (#233) 
* fix

* test
 2023-12-20 10:44:01 -08:00
Angelos Katharopoulos
2807c6aff0
Implements divide for integer types and adds floor_divide op (#228) 
* Add floor_divide
* Add floor_divide to the tests
* Add floor_divide to the docs
 2023-12-19 20:12:19 -08:00
davidkoski
de892cb66c
fix for non-macos build issue on cblas.h (#227) 2023-12-19 17:01:59 -08:00
davidkoski
37024d899c
fixes for building with swiftpm (#225) 
- clbas is part of veclib (compile failure)
- add SWIFTPM_BUNDLE #define to allow loading the metallib from a swiftpm resource bundle
 2023-12-19 16:22:10 -08:00
Diogo
137f55bf28
fail early if readinto does not exist (#221) 2023-12-19 13:27:17 -08:00
Emircan Erol
e549f84532
Triplet Loss (#211) 
* Triplet Loss

* Requested Changes

* Margin to alpha
 2023-12-19 12:37:12 -08:00
Angelos Katharopoulos
dfa9f4bc58
An initial quantized matmul implementation (#205) 
* Add quantized matvec
* Add quantized matrix matrix with 2nd matrix transposed
* Add quantized matmul tests
* Add a slow cpu quantized matmul
* Add a slightly faster vectorized cpu version
 2023-12-18 23:18:57 -08:00
Abe Leininger
e6872a4149
Added linspace (#181) 
* linspace ops support

---------

Co-authored-by: Awni Hannun <awni@apple.com>
 2023-12-18 19:57:55 -08:00
Juarez Bochi
f4f6e17d45
Fix cross-attention (#210) 
* Fix cross-attention

With the current code, ln2 is a no-op. Its output should be passed to the cross-attention layer

* Add name to contributors
 2023-12-18 12:27:27 -08:00