Quantprobe Runs a 110B-Parameter LLM on a 16GB RAM 2016 Desktop

Quantprobe, a new open-source tool built on llama.cpp, argues that where model weights sit across memory tiers (VRAM, RAM, disk) matters more than how much memory you have. Tested entirely on a 2016 desktop (GTX 1060 6GB, 16GB DDR4, SATA SSD), the project streamed GLM-4.5-Air 110B from a SATA drive and hit 0.19 tok/s — landing inside a 0.2-0.3 tok/s range the author pre-registered before running the hardware. A hybrid VRAM/RAM placement for Qwen3-30B-A3B reached 19.3 tok/s, again matching a prediction made before measurement.

The work distills four 'placement laws': incoherence rotation helps only full-rank tensors and wrecks low-rank ones; trained networks are dense everywhere, making 2-bit quantization a practical floor; fragility varies unpredictably between architectures and can only be found via a short functional probe; and decode speed across model sizes follows a single tiered-bandwidth equation. A control experiment using two byte-identical GGUF files that differ only in layer placement produced a 2.25 perplexity gap, backing the core claim with hard evidence rather than intuition.

For engineers, the practical payoff is a CLI that plans optimal placement and predicts throughput for any model/machine combo, with 16 built-in hardware presets and no GPU auto-detection required. Every claim ships with reproducible scripts, logs, and honestly documented dead ends (dynamic top-k, semantic paging), making the methodology as notable as the numbers themselves.