Single MI300X, Real Coding Agents: DeepSeek V4 Flash Throughput Tested

Instead of relying on synthetic benchmark traffic, an engineer measured what a single AMD MI300X GPU actually delivers when serving DeepSeek V4 Flash to autonomous coding agents working on real open-source repositories. Using an open-source vLLM-based serving stack, the tests found the GPU comfortably handles 32 concurrent agents — but beyond that point, performance doesn't degrade gracefully, it collapses.

In the tuned 32-agent configuration, the setup sustained 582 tokens/sec generation and 78.3K tokens/sec prompt processing, with a 93% prefix cache hit rate acting as the load-bearing metric — essential because agents resend their entire growing conversation history every turn. AITER GEMM tuning, combined with scheduler and kernel-lookup fixes, added an 18% throughput gain by eliminating fallback to untuned kernel shapes.

On cost, at $2/hour for the box, generated tokens run about $0.95 per million, while cache-served prompt tokens are nearly free. But pushing to 48 agents overflows the GPU's KV cache, cache hit rate drops below 30%, and throughput collapses rather than degrading smoothly — meaning capacity planning should be sized to cache capacity, not queue depth.