Getting Started¶

Run your first LLM on Apple silicon in under five minutes.

1. Install Ollama¶

curl -fsSL https://ollama.com/install.sh | sh

Ollama installs as a launchd service and auto-starts. Verify: ollama --version.

See the Ollama Setup Guide for environment variables, dual-stack architecture, and macOS troubleshooting.

ollama pull gemma4:26b-nvfp4

RAM	Recommended Model
16 GB	`llama3.1:8b` or `qwen3-coder:8b`
32 GB	`gemma4:26b-nvfp4` or `qwen3-coder:8b`
64 GB	`gemma4:26b-nvfp4`, `qwen3-coder:30b`, or `deepseek-r1:32b`
128 GB+	`deepseek-r1:70b` or `qwen3-coder:30b`

See the Model Selection guide for detailed recommendations.

ollama run gemma4:26b-nvfp4 "What is the capital of France?"

For API usage, see the Ollama API docs.

pip install open-webui
open-webui serve

Open http://localhost:8080 — your Ollama models appear automatically. See the Open WebUI Guide for MLX server integration and tool-calling patches.

MLX is Apple's native ML framework — ~29% faster than Ollama for single-batch inference on M1 Max.

pip install omlx
omlx serve --model mlx-community/gemma-4-26b-it-4bit --port 8000

Engine	Best For	Install
oMLX	Chat, agents, RAG — fastest prefill (5x faster than mlx_lm)	`pip install omlx`
Rapid-MLX	Long-form generation — fastest generation (38% faster than mlx_lm)	`pip install rapid-mlx`
mlx_lm	Stable fallback, most documented	`pip install mlx-lm`

See the MLX Setup Guide for engine comparison, PYTHONPATH fix, and macOS troubleshooting.

Ollama is running: curl http://localhost:11434/api/tags returns JSON
Model responds: ollama run gemma4:26b-nvfp4 "Hi" generates text
Open WebUI loads: http://localhost:8080 shows the chat interface
Metal acceleration: GPU utilization visible in Activity Monitor during inference