Google DeepMind has found a new way to build text diffusion models without training them from the ground up. The company converted the finished Gemma 4 into a diffusion model using less than 10 percent of the original training budget. The model is called DiffusionGemma and was released in June 2026; now the technical report has followed.
The essentials
- DiffusionGemma generates 256 tokens in parallel instead of sequentially, achieving about 1,500 tokens per second on an Nvidia H100
- Converting existing models requires only <10% of the original training budget
- Bidirectional reasoning: the model can correct errors during generation, not just afterward
- Quality lags behind the original, especially on reasoning tasks – but speed is several times higher
How does the conversion work?
Instead of training a new model, Google uses two training phases. In the first step, the model learns to reconstruct noisy text blocks from example data – similar to how image AI models generate a picture from noise. Then comes a combined phase of reinforcement learning and sampler distillation, which Google calls SD·RL. Reinforcement learning improves answer quality, while sampler distillation lets the model work with fewer compute steps.
This combined approach raises quality on reasoning benchmarks by an average of 10 points while nearly quadrupling tokens per compute step. Side effect: DiffusionGemma's answers are about 50 percent shorter – which further boosts speed.
Self-correction instead of linear thinking
Autoregressive models like Gemma 4 must commit to the first digit of an answer before they've worked through the reasoning. Google's report shows an example: Gemma 4 writes "-1" first, realizes during the derivation that "-25" is correct, and adds a correction afterward.
DiffusionGemma develops answer and reasoning in parallel. The model can fix mistakes before the output is finalized. With Sudoku puzzles, the advantage is clear: after minimal fine-tuning, DiffusionGemma solves nearly 85 percent of puzzles correctly, while the base model fails at the task entirely. Structured outputs like JSON or code repairs finish after just 2–3 refinement steps, because the input already determines most tokens.
| Aspect | DiffusionGemma | Gemma 4 (autoregressive) |
|---|---|---|
| Throughput | ~1,500 tokens/s | Significantly slower |
| Quality (benchmarks) | Behind original | Higher |
| Reasoning ability | Bidirectional, self-correcting | Linear, correctable only afterward |
| Training budget | <10% of original | 100% |
Where are the limits?
Absolute performance falls short of the autoregressive base model – especially on demanding reasoning tasks. Google cites several reasons: DiffusionGemma was not trained as a diffusion model from scratch, but converted from an existing model. This has consequences for quality. Multi-user scenarios are also still limited.
One advantage remains: DiffusionGemma can switch between both modes – users can choose between parallel and sequential generation depending on the task.
What does this mean for businesses?
The approach significantly lowers the barrier to entry for text diffusion models. If you already have a base model, you can convert it into a faster model with 90 percent less compute – relevant for organizations with limited GPU budgets. For applications that prioritize speed over maximum quality (real-time systems, mass processing), DiffusionGemma could be interesting. However, it remains unclear whether this approach works for specialized models – developers would need to test it themselves.
Sources
Editorially owned by Ideal Syka. Sources and method: Newsroom & method. Tips and corrections: ai@i6eal.de.
