zhangyiss/mlx
1
0
Fork 0
mirror of https://github.com/ml-explore/mlx.git synced 2025-06-25 18:11:15 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Do not release buffers on exit (#1142)

Browse Source
This commit is contained in:
Cheng 2024-07-16 07:12:24 +09:00 committed by GitHub
parent 987785d8d7
commit 2f83d6e4b7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 11 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
mlx/backend/metal/allocator.cpp
View File

@ -242,8 +242,17 @@ void MetalAllocator::free(Buffer buffer) {
}
MetalAllocator& allocator() {
static MetalAllocator allocator_;
return allocator_;
// By creating the |allocator_| on heap, the destructor of MetalAllocator will
// not be called on exit and all the buffers will be leaked. This is necessary
// because releasing buffers can take more than 30sec when the program holds a
// lot of RAM (for example inferencing a LLM), and it would feel frozen to
// users when exiting.
// TODO(zcbenz): Consider using the `base::NoDestructor` class from Chromium
// when applying this pattern to more places, or when introducing sanitizers
// to MLX.
// https://source.chromium.org/chromium/chromium/src/+/main:base/no_destructor.h
static MetalAllocator* allocator_ = new MetalAllocator;
return *allocator_;
}
size_t set_cache_limit(size_t limit) {