mlx-examples/llms/mlx_lm/__init__.py

# Copyright © 2023-2024 Apple Inc.

import os

from ._version import __version__

os.environ["TRANSFORMERS_NO_ADVISORY_WARNINGS"] = "1"

from .utils import convert, generate, load, stream_generate


def get_estimate_mem():
    from .estimate_memory import estimate_mem

    return estimate_mem
Fix import warning (#479) * fix import warning * fix version import * remove api, move convert to utils * also update circle to run external PRs 2024-02-28 00:47:56 +08:00			`# Copyright © 2023-2024 Apple Inc.`
bump version and include in package (#475) 2024-02-22 01:40:36 +08:00
Fix detokenizer space match for quote (#1072) * fix + test * remove transformer flax/torch warning * format 2024-10-28 06:06:07 +08:00			`import os`

Make sure to import the correct "version" module when installing mlx_whisper and mlx_lm from local source code. (#969) * Make sure to import the correct "version" module when installing the mlx_whisper package from local source code. * Make sure to import the correct "version" module when installing the mlx_lm package from local source code * fix --------- Co-authored-by: Awni Hannun <awni@apple.com> 2024-09-04 04:16:21 +08:00			`from ._version import __version__`
Fix detokenizer space match for quote (#1072) * fix + test * remove transformer flax/torch warning * format 2024-10-28 06:06:07 +08:00
			`os.environ["TRANSFORMERS_NO_ADVISORY_WARNINGS"] = "1"`

mlx_lm: Add Streaming Capability to Generate Function (#807) * Add streaming feature to text generation function * separate stream and regular functions --------- Co-authored-by: Awni Hannun <awni@apple.com> 2024-06-04 00:04:39 +08:00			`from .utils import convert, generate, load, stream_generate`
Add memory estimation tool for MLX language models This commit introduces a comprehensive memory estimation utility for MLX language models, supporting: - Dynamic parameter calculation across diverse model architectures - Handling of quantized and standard models - Estimation of model weights, KV cache, and overhead memory - Support for bounded and unbounded KV cache modes - Flexible configuration via command-line arguments The new tool provides detailed memory usage insights for different model configurations and generation scenarios. 2025-03-10 10:59:09 +08:00

			`def get_estimate_mem():`
			`from .estimate_memory import estimate_mem`

			`return estimate_mem`