2025-11-15 · 3 min read

LoRA on Apple Silicon with MLX

MLX is Apple's machine learning framework for M-series chips. It enables GPU-accelerated training and inference without external hardware. LoRA training on an M2 Pro is practical and fast.

Why MLX

MLX provides: - Unified memory (GPU and CPU share the same memory pool, no constant transfers) - Native M-series optimization (faster than PyTorch MPS in many cases) - Low compile overhead (good for iterative work) - Simplified distributed training on multiple GPUs (M1 Ultra, M2 Max with multiple cores)

Trade-off: smaller ecosystem than PyTorch. Some advanced techniques are missing.

Installation

pip install mlx mlx-lm

Converting HuggingFace checkpoint to MLX

python -m mlx_lm.convert \
  --hf-path Qwen/Qwen2.5-7B-Instruct \
  --mlx-path ./qwen25-7b-mlx \
  --quantize

The --quantize flag converts to 4-bit (or specify --quantize int8, --quantize fp16).

Result: a folder with MLX-native weights, typically 20–30% smaller than safetensors.

Training a LoRA

python -m mlx_lm.lora \
  --model ./qwen25-7b-mlx \
  --train \
  --data data/support_pairs.jsonl \
  --iters 500 \
  --batch-size 2 \
  --learning-rate 2e-4 \
  --lora-dims 8

Real M2 Pro example: 500 iterations on 200 support examples takes ~8 minutes (wall-clock, not GPU time).

Performance by M-series chip

Chip	Max RAM	Typical LoRA training time	Notes
M2 (8 GB)	8 GB	15–20 min (3B model)	Tight, only Q-bit models
M2 Pro (16 GB)	16 GB	8–10 min (7B model)	Comfortable LoRA training
M2 Max (32 GB)	32 GB	5–7 min (7B model)	Fast, can do larger ranks
M2 Ultra (128 GB)	128 GB	2–3 min (70B model)	Enterprise-grade, rare

M2 Pro is the sweet spot for individual developers. M2 Max is overkill unless you're training multiple adapters concurrently.

Inference

from mlx_lm import load, generate

# Load quantized model + LoRA
model, tokenizer = load("./qwen25-7b-mlx")

# LoRA is automatically applied if in the model directory
response = generate(
    model,
    tokenizer,
    prompt="The system is down. ",
    max_tokens=50,
    temperature=0.7
)

print(response)

Throughput on M2 Pro: 15–20 tokens/second, similar to llama.cpp on CPU but with higher quality (model is fp16, not quantized).

Exporting to other formats

After training, export to safetensors for use in other ecosystems:

python export_lora.py --mlx-lora ./lora-output --output ./lora.safetensors

Then load in Diffusers, vLLM, or Ollama.

Hybrid: train on MLX, deploy elsewhere

Train on MacBook Pro (fast, free)
Export to safetensors
Upload to ModelForgeLab registry
Serve on cloud GPU (vLLM) or CPU (llama.cpp)

This workflow is common for small teams: iterate fast locally, deploy to production infrastructure.

Memory-constrained training

M2 with 8 GB is tight. Options:

# Use smaller model
python -m mlx_lm.lora \
  --model Phi-3-mini \  # 3.8B instead of 7B
  --train \
  --data data.jsonl \
  --batch-size 1 \  # reduce batch
  --lora-dims 4  # smaller rank

Phi-3-mini fits comfortably on 8 GB and trains in 5–8 minutes for 200 examples.

Known limitations

Architecture support: MLX supports Llama, Qwen, Mistral, Phi. Older architectures may fail.
Quantization during training: Only post-training quantization supported; cannot QLoRA-train.
Distributed training: MLX can split across multiple GPUs on one machine, but not across networks.

These are minor; for most use cases, MLX is competitive with PyTorch.

Real-world example: phi3-mini-sql-helper

# Convert
python -m mlx_lm.convert \
  --hf-path microsoft/Phi-3-mini-4k-instruct \
  --mlx-path ./phi3-mlx \
  --quantize int8

# Train
python -m mlx_lm.lora \
  --model ./phi3-mlx \
  --train \
  --data sql_examples.jsonl \
  --iters 300 \
  --batch-size 2 \
  --lora-dims 8

# Inference
python -c "
from mlx_lm import load, generate
model, tokenizer = load('./phi3-mlx')
print(generate(model, tokenizer, prompt='SELECT * FROM users WHERE ', max_tokens=30))
"

On M2 Pro: 5 minutes training, 100 tokens/second inference. Development loop is tight.

MLX vs PyTorch on MacBook

Aspect	MLX	PyTorch MPS
Throughput (training)	100%	70–80%
Throughput (inference)	100%	80–90%
Ease of setup	Simple	Requires MPS plugin
Community	Growing	Mature
Quantization training	No	No (QLoRA via bitsandbytes)

MLX is faster and simpler. PyTorch MPS is more mature if you need cutting-edge techniques.

When to use MLX

You own an M-series Mac and want to train locally
You want the fastest iteration cycle (8–15 minutes per experiment)
You're training small adapters (rank 8–16)
You plan to export and deploy elsewhere

MLX removes the barrier to local training. A 7B LoRA trains faster on an M2 Pro than on a Tesla V100 in the cloud, with zero infrastructure cost.