DiffusionGemma: Google's Diffusion-Based LLM Developer Guide
Google's DiffusionGemma developer guide: diffusion-based LLM with bidirectional attention, 4x faster generation, vLLM support, and Sudoku fine-tuning results.
Google has released a developer guide for DiffusionGemma, an experimental text-diffusion model built on the Gemma 4 backbone. Instead of generating tokens one by one, it denoises a 256-token canvas in parallel using bidirectional attention, shifting the bottleneck from memory bandwidth to compute. This yields up to 4x faster generation, with reported throughput above 700 tokens/sec on an RTX 5090 and over 1000 tokens/sec on a single H100.
The model is structured as a 26B-parameter Mixture-of-Experts network that activates only 3.8B parameters per inference step, enabling quantized deployment within 18GB of VRAM. For sequences beyond 256 tokens, a block-autoregressive scheme commits each denoised block to a KV cache before starting the next canvas, combining diffusion parallelism with the stability of autoregressive decoding.
Google demonstrated fine-tuning via the Hackable Diffusion JAX toolbox on Sudoku solving, a task where autoregressive models typically fail due to left-to-right generation. The base model solved 0% of puzzles, but after supervised fine-tuning, accuracy jumped to 80%, with the tuned model also converging in fewer denoising steps thanks to early-stopping behavior.
Serving support has been added directly to vLLM, with the model released under Apache 2.0 on Hugging Face, alongside support across Transformers, SGLang, MLX, Unsloth, NVIDIA NeMo, and NIM/Model Garden deployment paths.