Unified Memory: Why Mini PCs Run 70B Models a Big GPU Can't
Unified-memory mini PCs like AMD's Strix Halo can load 70B-parameter models that a $2,000 RTX 5090 cannot fit. Here's why capacity and bandwidth pull in opposite directions.
Two $2,000 machines expose a real paradox: an RTX 5090 with 32GB of the fastest consumer VRAM can't even load a 4-bit quantized 70B model, while a modest mini PC with 128GB of unified memory loads it fine and runs it, just slowly. The trick is unified memory architecture, where CPU, integrated GPU and NPU share one LPDDR5X pool instead of a hard-capped separate VRAM block, letting almost the entire pool go to the model.
That capacity comes at a cost: bandwidth. Per the roofline model, token generation is bandwidth-bound because nearly the whole model must be streamed from memory for every token produced. Devices like Strix Halo, at roughly 256 GB/s, lag far behind GPUs pushing 900-1,800 GB/s of GDDR bandwidth. Mixture-of-Experts models sidestep this by activating far fewer parameters per token, delivering a dramatic speedup on the same hardware.
The less-discussed bottleneck is prompt processing (prefill), which is compute-bound rather than bandwidth-bound. Because integrated GPUs have a fraction of a discrete card's compute throughput, long documents or codebases can take tens of seconds just to be read before generation even starts. The large NPU TOPS figures marketed on these chips are largely irrelevant here too, since NPUs share the same memory pool and don't move the fundamental decode bottleneck.