DFlash is an open-source lightweight block-diffusion model designed for speculative decoding that replaces sequential token drafting with block-parallel proposals. A small, fast drafter produces an entire block of candidate future tokens in a single forward pass; the larger target model then verifies those candidates in parallel and accepts the longest valid prefix. This converts some of the sequential token-generation work into parallel GPU tasks while preserving the target model’s output distribution.
Throughput gains on NVIDIA Blackwell
According to the source, DFlash increases inference performance for gpt-oss-120b on NVIDIA Blackwell by up to 15× at the same interactivity level in a TensorRT-LLM run on an eight-GPU NVIDIA DGX B300 system. In the high interactivity range of 500–600 tokens/sec per user, DFlash delivered more than 15× throughput improvement over autoregressive decoding on Blackwell, and it outperformed EAGLE-3 speculative decoding by a factor of 1.5. At the lowest concurrency point (batch size 1), DFlash more than doubled interactivity on Blackwell.
The method is particularly valuable in decode-constrained regions where LLM inference is often limited by memory movement and the sequential nature of token generation rather than raw compute. By moving part of that work into parallel block drafting and parallel verification, DFlash can leverage more of Blackwell’s compute capacity (the source states Blackwell Ultra provides 15 PFLOPS of dense NVFP4 compute) and serve more users concurrently at the same interactivity target.
NVIDIA Blackwell Ultra GPUs combine two reticle-sized dies connected by 10 TB/s chip-to-chip interconnect, forming a unified domain with 160 SMs and 640 fifth-generation Tensor Cores — an architecture that benefits from the additional parallelism exposed by DFlash.
Comparison with EAGLE-3 and smaller models
DFlash shows interactivity speedups over EAGLE-3 across datasets and model sizes. For Llama 3.1 8B Instruct, DFlash nearly doubled performance versus EAGLE-3 on the Speed-Bench multilingual dataset. Summary figures from the source show average speedups at matched user concurrency levels as follows: for gpt-oss-120b EAGLE-3 averages 1.7× while DFlash averages 2.3×; for Llama 3.1 8B Instruct EAGLE-3 averages 2.2× while DFlash averages 2.8×.
Selected per-task comparisons (source data, matched concurrency):
- Coding: gpt-oss-120b — EAGLE-3: 1.8×, DFlash: 2.6×; Llama 3.1 8B — EAGLE-3: 2.3×, DFlash: 3.0×
- Multilingual: gpt-oss-120b — EAGLE-3: 1.8×, DFlash: 2.6×; Llama 3.1 8B — EAGLE-3: 1.4×, DFlash: 2.4×
Rapid availability for developers in the NVIDIA ecosystem
Researchers at UC San Diego published "DFlash: Block Diffusion for Flash Speculative Decoding" in February 2026 as part of ongoing work on faster LLM inference for NVIDIA Blackwell. Implemented in PyTorch with native CUDA support, DFlash improves decode performance via block-diffusion speculative decoding. The research team and the open-source inference community ensured framework support across SGLang and vLLM so that developers have a clear path to adopt DFlash in existing serving stacks.
Since the paper’s release, the team published 20 DFlash model checkpoints on Hugging Face with Blackwell and Hopper recipes, covering model families including Qwen, Kimi K2.6, Llama, Gemma, and gpt-oss. The recipes include support for SGLang and vLLM.
On vLLM, developers can swap EAGLE-3 with a DFlash checkpoint without code changes outside of configuration; the integration uses the open-source Speculators library to connect the DFlash drafter to the target model’s hidden states inside the vLLM inference path on NVIDIA GPUs. The source reports that on Gemma 4 31B running on a single Blackwell Ultra GPU this path delivered up to 5.8× higher throughput at the same concurrency over autoregressive decoding across math, coding, and chat benchmarks (Math500, GSM8K, HumanEval, MBPP, MT-Bench).
For SGLang, moving from EAGLE to DFlash requires updating the speculative decoding algorithm to DFlash and providing the matching DFlash draft checkpoint. On Qwen3 8-B running on a single Blackwell GPU with SGLang, the source reports up to 5.1× throughput at the same concurrency over autoregressive decoding on Math500 and 4.2× on HumanEval.
This early, broad model and framework coverage on NVIDIA GPUs matters because it allows teams to evaluate and deploy DFlash optimizations through the frameworks they already use, without application refactoring.
How DFlash speculative decoding works
Speculative decoding has two phases: drafting and verification. A smaller draft model proposes future tokens; the target model verifies those tokens in parallel and accepts the longest valid prefix. If the draft is correct, the system produces multiple tokens with a single target-model verification pass.
Traditional speculative decoding often uses autoregressive drafters that still generate tokens sequentially, limiting how far throughput can be pushed. DFlash replaces that drafter with a lightweight block-diffusion drafter that predicts a block of masked future tokens in one forward pass.
DFlash combines three key techniques:
- Block-diffusion drafting: the drafter predicts multiple future tokens in parallel.
- Target hidden-state conditioning: the drafter uses context features extracted from the target model.
- KV injection: target context features are injected into the draft model’s key-value projections across layers to maintain high acceptance rates.
Because the target model still performs verification, DFlash preserves the target model’s output distribution while accelerating generation.
Getting started
DFlash is available as open model checkpoints on NVIDIA GPUs and is supported in SGLang, vLLM, and TensorRT-LLM. The combination of released checkpoints, recipes, and framework integrations provides a practical route for teams to test and deploy DFlash to improve inference throughput and interactivity on NVIDIA hardware.
