zhangyiss/mlx
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx.git synced 2025-09-26 15:58:14 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Lower sorted QMM gather threshold (#2609)

Browse Source
This commit is contained in:
Awni Hannun
2025-09-19 18:22:55 -07:00
committed by GitHub
parent 787c0d90cd
commit ec2ab42888
1 changed files with 2 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
mlx/backend/metal/quantized.cpp
View File

@@ -948,8 +948,8 @@ void GatherQMM::eval_gpu(const std::vector<array>& inputs, array& out) {
// We are walking x in order and w is also in order so we can batch up the
// matmuls and reuse reading x and w.
//
// TODO: Tune 16 and 8 here a bit better.
if (M == 1 && B >= 16 && right_sorted_ == true && B / E >= 8) {
// TODO: Tune 16 and 4 here a bit better.
if (M == 1 && B >= 16 && right_sorted_ == true && B / E >= 4) {
gather_qmm_rhs(
x,
w,