Google DeepMind Details DiffusionGemma Retrofit Using Under 10 Percent of Original Training Budget
Google DeepMind has shown that text diffusion models need not be trained from scratch. Researchers retrofitted Gemma 4 into DiffusionGemma using less than ten percent of the original Gemma training budget, a materially cheaper path than building a parallel diffusion stack from zero. The resulting model generates two hundred fifty-six tokens in parallel rather than one at a time, reaching roughly fifteen hundred tokens per second on an Nvidia H100 GPU. That throughput profile favors latency-sensitive applications where parallel decoding matters more than peak reasoning quality. Reporting attributes the result to a retrofit rather than a greenfield diffusion architecture, which lowers the barrier for labs that already hold strong autoregressive checkpoints. The main caveat in the evidence is explicit: DiffusionGemma still trails the autoregressive Gemma base on reasoning benchmarks, so the speed gain does not yet translate into reasoning parity.
Google DeepMind Details DiffusionGemma Retrofit Using Under 10 Percent of Original Training Budget
Google DeepMind retrofitted Gemma 4 into a diffusion model using less than 10 percent of the original training budget. DiffusionGemma generates 256 tokens in parallel instead of one at a time, hitting about 1,500 tokens per second on an Nvidia H100, though quality still trails the autoregressive base on reasoning benchmarks.
Key takeaway
Retrofitting Gemma 4 into DiffusionGemma under 10% of the original training budget shows text diffusion can piggyback on existing autoregressive checkpoints rather than demanding a full from-scratch training run.
What happened
According to The Decoder, Google DeepMind retrofitted Gemma 4 into a diffusion model called DiffusionGemma rather than training a new architecture from scratch, using less than 10 percent of the original Gemma training budget.
The Decoder reports DiffusionGemma generates 256 tokens in parallel instead of one at a time, hitting about 1,500 tokens per second on an Nvidia H100, though quality still trails the autoregressive base on reasoning benchmarks.
Evidence
Google DeepMind retrofitted Gemma 4 into a diffusion model using less than 10 percent of the original training budget.
The Decoder · attributed
Instead of training a new model from scratch, Google DeepMind retrofitted Gemma 4 into a diffusion model using less than 10 percent of the original training budget.
DiffusionGemma generates 256 tokens in parallel and reaches about 1,500 tokens per second on an Nvidia H100.
The Decoder · attributed
DiffusionGemma generates 256 tokens in parallel instead of one at a time, hitting about 1,500 tokens per second on an Nvidia H100
DiffusionGemma quality still trails the autoregressive Gemma base on reasoning benchmarks.
The Decoder · attributed
though quality still trails the autoregressive base on reasoning benchmarks.
Why it matters
The approach could reshape deployment economics for labs that want parallel token generation without abandoning pretrained Gemma weights, but reasoning quality gaps mean it is not a drop-in replacement yet.
Limits and uncertainties
Reporting states DiffusionGemma still trails the autoregressive Gemma base on reasoning benchmarks, so throughput gains do not imply reasoning parity.
The evidence packet contains a single attributed article and does not include independent benchmark tables or Google DeepMind primary documentation.
Practical implications
Teams holding Gemma 4 checkpoints may evaluate a diffusion retrofit path that reportedly costs under 10 percent of the original training budget instead of training a diffusion model from scratch.
Operators optimizing for decode latency on Nvidia H100 hardware may weigh parallel 256-token generation at roughly 1,500 tokens per second against lower reasoning scores.
Builders targeting reasoning-heavy workloads may need to keep the autoregressive Gemma base until reported benchmark gaps close.
What to watch
Whether Google DeepMind or third parties publish additional reasoning benchmarks comparing DiffusionGemma to the autoregressive Gemma 4 base.
Whether similar retrofit diffusion releases appear for other open autoregressive model families beyond Gemma 4.
