Frontier-class LLM Inference on a Laptop CPU
cpubrrr surpasses llama.cpp on Apple M4 Max CPU for frontier-class LLMs.
The new research runtime cpubrrr outperforms llama.cpp's CPU path by running frontier-class mixture-of-experts models solely on an Apple M4 Max CPU. It operates without GPU support, linking only to the C standard library, and can run four different MoE models simultaneously. Its performance is superior to llama.cpp in both MXFP4 and Q4_K formats.