MLX Engine Comparison¶

Recommendation: Use oMLX for interactive work (chat, agents, RAG) — its 5x faster prefill makes it feel dramatically snappier. Use Rapid-MLX for long-form generation (batch processing, document writing). Use mlx_lm as the stable fallback.

A head-to-head comparison of the three major MLX inference engines running the same model (Qwen 3.6 35B-A3B-abliterated, 4bit-MLX quantization) on the same hardware.

Test setup¶

• Model: Qwen 3.6 35B-A3B-abliterated (4bit-MLX)
• Hardware: MacBook Pro M1 Max (64 GB)
• Prompt: 2,000 tokens of structured context
• Generation: 512 tokens
• Measurement: Each engine tested 5 times, median reported

Overall results¶

Per-turn breakdown¶

The following table shows how each engine performs across a typical interaction lifecycle:

When to use each engine¶

mlx_lm — The Balanced Choice¶

Use mlx_lm for: General use, development, and most production workloads.

• Good balance of prefill and generation performance
• Most mature and well-documented engine
• Strong community support and frequent updates
• Effective throughput of 123.87 tok/s — excellent for most workloads
• 3.86s cold start is acceptable for interactive use

# Install
pip install mlx-lm

# Serve
mlx_lm.server --model Qwen3.5-35B-A3B-abliterated-4bit

Rapid-MLX — The Generation Speed Demon¶

Pick Rapid-MLX for: Long-running generation sessions where prefill happens once.

• Fastest generation throughput at 51.4 tok/s — 38% faster than mlx_lm
• Very high prefill overhead (~28s for 2K tokens) — 75% slower than mlx_lm
• Effective throughput of only 18.3 tok/s due to prefill penalty
• Ideal for: long document generation, batch processing, or streaming where you generate thousands of tokens per prompt

# Install
pip install rapid-mlx

# Serve
rapid-mlx serve --model Qwen3.5-35B-A3B-abliterated-4bit

oMLX — The Prefill Champion¶

oMLX excels at: Interactive chat, RAG pipelines, and short-turn workloads.

• Fastest prefill at ~3s for 2K tokens — 5x faster than mlx_lm
• Effective throughput of 170.72 tok/s — 38% faster than mlx_lm
• Generation throughput of 39.16 tok/s — competitive with mlx_lm
• Fastest cold start at 3.50s
• Ideal for: chatbots, multi-turn conversations, agent loops, and any workload with frequent prompt changes

# Install
pip install omlx

# Serve
omlx serve --model Qwen3.5-35B-A3B-abliterated-4bit

Decision matrix¶

API overhead comparison¶

All three engines provide OpenAI-compatible APIs. The overhead of /v1/chat/completions vs /api/generate is consistent across engines at 33–37%:

Installation comparison¶

Recommendation¶

Start with oMLX for interactive use cases — its prefill performance makes it feel dramatically faster in practice. Switch to Rapid-MLX if you are doing long-form generation. Use mlx_lm as the stable fallback if you encounter issues with either alternative.