DeepSeek V4 Flash reaches 32 tok/s on AMD Ryzen AI MAX+ 395

The Lucebox project ran the full 284B-parameter DeepSeek V4 Flash mixture-of-experts model entirely locally on an AMD Ryzen AI MAX+ 395 APU, using its 128 GB unified LPDDR5X memory pool. With no discrete GPU or remote inference service involved, the setup achieved up to 32.0 tok/s decode and roughly 250 tok/s with indexed sparse prefill. Submitted to the LocalMaxxing benchmark site, the result beat the next-best entry for the same hardware, HipFire at 18.99 tok/s, by 68.5%, and reached 2.05x the previous best (DwarfStar at 15.6 tok/s).

The speed comes from ROCmFPX, a family of block-based quantization formats built specifically for AMD's ROCm/HIP path. A mixed-precision recipe packs the largest routed-expert matrices at 2 bits, expert projections at 3.5 bits, and more sensitive layers at 4.25 bits, compressing the model to 102.3 GB, roughly 2.88 bits per parameter. A compact draft model called DSpark adds speculative decoding, verifying multiple tokens per pass and lifting throughput 26.4% over plain autoregressive decode.

For engineers, the result demonstrates that consumer-grade unified-memory APUs can now serve 284B-parameter MoE models at usable speeds without a second machine or cloud GPU. The code is public on the Lucebox main branch and reproducible, though sparse prefill changes floating-point reduction order and the 4-expert routing mode trades some quality margin for speed compared to the model's default 6-expert setting.