Apple Silicon macOS VMs Get 11-16x Faster LLM Inference via Llama.cpp
Cua's Metal capability shim delivers 11-16x faster llama.cpp LLM inference in macOS VMs on Apple Silicon, with open benchmarks and source.
Cua has published a small research release that substantially improves LLM inference throughput inside macOS virtual machines built on Apple's Virtualization.framework. The root cause: guest VMs report an outdated Apple GPU family and a 32 KB threadgroup memory limit, which disables Metal fast paths like SIMD-group matrix, SIMD-group reduction, and bfloat16 — forcing llama.cpp onto a slower kernel path even though the underlying physical GPU could run faster ones.
Their fix is a process-scoped Metal capability shim that intercepts selected capability queries inside one guest process, reporting Apple GPU family 9 (1009) and 64 KB of threadgroup memory. It's not physical GPU passthrough — no hardware assignment or kernel changes are involved, just corrected capability answers that let Metal applications choose better kernels.
On an M1 Ultra, TinyLlama 1.1B saw prompt processing improve 11.08x and token generation improve 16.36x versus a stock VM, reaching 98.25% and 72.06% of bare-metal speed respectively. Google's Gemma 4 12B QAT Q4_0 model improved 7.20x on prompt processing and 14.54x on generation, hitting 99.59% and 94.82% of bare-metal performance. MLX-LM showed no change since it was already near-optimal in the stock VM.
The project is released under the same permissive license as Lume/Cua, with source, build scripts, a capability probe, and raw benchmark logs included so results can be independently reproduced across different Apple Silicon chips, macOS versions, and Metal workloads.