DFlash in llama.cpp: 4.44x Faster Local Inference on Qwen 3.6 27B

DFlash, freshly merged into llama.cpp, accelerates speculative decoding using a block-diffusion drafter from z-lab that fills up to 15 tokens per forward pass instead of drafting one token at a time. Tested on an RTX PRO 6000 Blackwell (96GB VRAM) with Qwen 3.6 27B, it reached 273 tok/s versus a 61 tok/s baseline at 36K context — a 4.44x speedup. Notably, the speedup grows with context length (1.44x at 512, 2.70x at 4K, 3.40x at 12K, 4.44x at 36K), the opposite of typical scaling behavior where models slow down as context grows.

On the same hardware, DFlash outperformed a previously tested MTP (multi-token prediction) approach at every draft length. Acceptance rates were similar (tau around 7.3 vs 6.7), but MTP requires one forward pass per drafted token, while the diffusion drafter fills an entire block in a single pass. This exploits the fact that decoding is memory-bandwidth bound rather than compute bound — the GPU mostly waits on weight reads, so amortizing that cost across accepted tokens pays off. Injecting the target model's hidden states into every drafter layer (KV injection) keeps the drafter accurate even across large blocks.

On quality, a 100-problem MATH-500 subset showed the base model scoring 87% versus DFlash's 86%, with identical results in 6 of 7 categories — architecturally it should be lossless under greedy decoding, and the single discrepancy is attributed to an early-stage implementation quirk. VRAM overhead was around 5GB (26GB vs 21GB baseline). Tests were run at concurrency 1, so behavior under high-throughput production serving remains unverified.