References
A curated index of official project sites, benchmark resources, research papers, model publishers, quantization references, and community tools for running LLMs on Apple silicon.
Official project sites
Ollama
MLX (Apple)
Hugging Face
Open WebUI
Monitoring & Diagnostics
| Resource | URL | Description |
| macmon | https://github.com/vladkens/macmon | Sudoless Apple silicon thermal and power monitoring — CPU/GPU frequency, temperature, power draw. |
| Activity Monitor | Built into macOS | Monitor GPU usage, memory pressure, and energy impact during inference. |
| Resource | URL | Description |
| CraftRigs | https://www.craftrigs.com | Community LLM benchmark database with Apple silicon results. Filter by model, quantization, and hardware. |
| WillItRunAI | https://willitrunai.com | Check if a model fits on your hardware. Enter model size and RAM to see feasibility. |
| LLMCheck | https://llmcheck.net | LLM performance database with Apple silicon-specific benchmarks. |
| EastonDev | https://eastondev.com | Apple silicon LLM benchmarks and guides. Regular updates with new model releases. |
| Vijay.eu | https://vijay.eu | Apple silicon ML benchmarks — token speeds, memory usage, and hardware comparisons. |
| r/LocalLLaMA | https://reddit.com/r/LocalLLaMA | The primary Reddit community for local LLM discussion. Extensive Apple silicon benchmark threads. |
Research papers & technical articles
Known Ollama issues (GitHub)
These are the most relevant open/closed issues for Apple silicon users:
Model publishers
These publishers provide pre-quantized models optimized for Apple silicon:
Quantization references
| Format | Description | Typical Size Reduction |
| GGUF | llama.cpp's native format. Supports many quantization levels (Q2_K through Q8_0). Most widely supported across tools. | 4×–8× vs. FP16 |
| NVFP4 | NVIDIA-style 4-bit floating point. High-quality quants for Apple silicon. | ~4× vs. FP16 |
| MXFP8 | Microscaling FP8 format. Emerging standard for 8-bit inference with minimal quality loss. | ~2× vs. FP16 |
| Q4_K_M | GGUF 4-bit quantization with medium importance grouping. Best quality-to-size ratio for most models. | ~4× vs. FP16 |
| Q8_0 | GGUF 8-bit quantization. Minimal quality loss, higher memory usage. | ~2× vs. FP16 |
| MLX 4-bit | MLX-native 4-bit quantization. Uses Apple's Metal Performance Shaders. | ~4× vs. FP16 |
| MLX 8-bit | MLX-native 8-bit quantization. Near-lossless quality. | ~2× vs. FP16 |
Quantization Selection Guide
| Your RAM | Max Model Size (FP16) | Recommended Quantization |
| 16 GB | 7B–8B | Q4_K_M or NVFP4 |
| 32 GB | 13B–15B | Q4_K_M or NVFP4 |
| 64 GB | 27B–30B | Q4_K_M, NVFP4, or MXFP8 |
| 128 GB | 70B+ | Q4_K_M or Q8_0 for smaller models |
| 192 GB | 120B+ | Q4_K_M for largest models |
Apple silicon hardware references
Identifying Your Hardware
# Chip type and model
sysctl -n machdep.cpu.brand_string
# Unified memory size (in bytes)
sysctl -n hw.memsize
# Memory bandwidth (approximate, in GB/s)
# M1: 68.25 (Pro/Max: 200/400)
# M2: 100 (Pro/Max: 200/400)
# M3: 150 (Pro/Max: 300/400)
# M4: 120 (Pro/Max: 240/480)
# GPU core count
system_profiler SPDisplaysDataType | grep "Total Number of Cores"
# Metal GPU family support
system_profiler SPDisplaysDataType | grep "Metal"
Memory Bandwidth by Chip
| Chip | Base (GB/s) | Pro (GB/s) | Max (GB/s) | Ultra (GB/s) |
| M1 | 68.25 | 200 | 400 | 800 |
| M2 | 100 | 200 | 400 | 800 |
| M3 | 150 | 300 | 400 | — |
| M4 | 120 | 240 | 480 | — |
Memory bandwidth is the key hardware factor for LLM inference speed on Apple silicon — it determines how fast model weights can be fed to the GPU.
# GPU statistics (power, utilization, temperature)
sudo powermetrics --samplers gpu_power -n 1 -i 1000
# Metal device info
system_profiler SPDisplaysDataType
These tools also support running LLMs on Apple silicon for different workflows:
| Platform | URL | Description |
| LM Studio | https://lmstudio.ai | GUI application for discovering, downloading, and running GGUF models. Built-in chat interface and local API server. |
| Jan | https://jan.ai | Open-source ChatGPT alternative that runs entirely offline. Supports multiple inference engines. |
| AnythingLLM | https://anythingllm.com | All-in-one AI desktop app with RAG support. Can use local models. |
| Khoj | https://khoj.dev | Open-source AI copilot that runs locally. Supports search, chat, and agent features. |
| LocalAI | https://localai.io | Self-hosted OpenAI API alternative. Supports multiple backends including llama.cpp and MLX. |
| llama.cpp | https://github.com/ggerganov/llama.cpp | The C/C++ inference engine that powers Ollama. Can be used directly for maximum control. |
| vLLM | https://github.com/vllm-project/vllm | High-throughput LLM serving engine. Limited Apple silicon support but improving. |
| ExLlamaV2 | https://github.com/turboderp/exllamav2 | Fast inference for Llama-family models. Primarily CUDA-focused but has experimental Metal support. |
Contributing
Found a broken link, missing resource, or new benchmark site? Open an issue or pull request on GitHub:
https://github.com/j3n0v4/apple-silicon-llm-guide
Last updated: August 2026. URLs and project statuses change over time. If a link is broken, please open an issue.