DeepSeek introduced DSpark, a speculative decoding module in which a small draft model proposes blocks of tokens that a frozen large language model verifies in a single pass. The company applied DSpark to its DeepSeek‑V4 family and published two checkpoints, DeepSeek‑V4‑Pro‑DSpark and DeepSeek‑V4‑Flash‑DSpark, which add the draft module to the unchanged preview weights of DeepSeek‑V4‑Pro and DeepSeek‑V4‑Flash. The modified models are available on Hugging Face under an MIT license.
Key idea
Speculative decoding works by having a lightweight drafter propose a block of tokens while the target model computes its own next‑token choices for every drafted position in one pass. The system keeps the longest run of drafted tokens that match the target model’s choices, speeding generation while verification preserves quality. DSpark targets three contributors to performance: the cost of drafting tokens, how many drafted tokens survive verification, and how much verification the large model performs. Its primary innovation is dynamically adjusting the third factor: verifying more when server load is light and less when load is heavy.
How it’s built
DSpark attaches a draft module to a frozen target model. DeepSeek trained only three parts of the draft: a drafting backbone, a small sequential component, and a confidence head; the module reuses the target model’s embedding layer and output head. For offline training, the team fed Open‑PerfectBlend prompts to the target model and trained the drafter to match the target’s token distribution and the confidence head to estimate each token’s acceptance odds.
The backbone is borrowed from DFlash, a prior parallel drafter: like DFlash, DSpark proposes tokens for every position in a block in a single pass. A parallel pass costs roughly the same regardless of block length, so parallel drafters can afford deeper backbones and stronger early guesses. Because each position is predicted independently, however, parallel drafters can splice together tokens from different valid continuations, degrading accuracy toward the end of a block.
To mitigate that, the authors added a compact sequential component called a Markov head, which adjusts each position’s token probabilities based on only the token drafted immediately before it (for example, after the drafter picks “of,” the likelihood of “course” increases while “problem” decreases). This step runs token by token but is small: increasing draft length from 4 tokens to 16 tokens adds only 0.2–1.3 percentage points to per‑round latency compared with the unmodified DFlash backbone.
For each drafted token, the confidence head predicts the conditional probability that the token will survive verification given that all earlier tokens in the block survived. These estimates tend to be overconfident, and choosing how many tokens to verify requires calibrated magnitudes rather than mere rankings. The team therefore applies a calibration step that rescales estimates position by position until chains of estimates match observed acceptance rates on held‑out data.
At serving time a scheduler multiplies per‑token confidence estimates to compute a survival probability for every possible draft length (a draft survives to length L only if every preceding token survives). The scheduler then selects a verification length for each request to maximize expected total output across users, consulting a startup‑measured profile of system speed at each load level. Under light traffic it verifies longer drafts to reduce latency; under heavy traffic it drops drafted tokens with low acceptance odds to free capacity for others.
Results
DeepSeek evaluated DSpark offline against retrained versions of earlier drafters on open‑weight models and in production against its previous serving setup. In all cases DSpark outperformed competing draft modules.
- DSpark increased the average number of tokens accepted per verification round. Versus the sequential drafter EAGLE‑3, DSpark improved accepted tokens by 30.9%, 26.7%, and 30.0% on Qwen3‑4B, Qwen3‑8B, and Qwen3‑14B, respectively. Versus the parallel drafter DFlash, gains were 16.3%, 18.4%, and 18.3% on the same models.
- The improvements also held on Gemma4‑12B, a model family separate from Qwen3, indicating the advantage is not limited to a single lineage.
- In production, DSpark made DeepSeek‑V4‑Flash generate tokens per user 60%–85% faster and DeepSeek‑V4‑Pro 57%–78% faster compared with DeepSeek’s previous production drafter MTP‑1, which proposed one token per cycle.
- When comparing throughput at different guaranteed per‑user token rates, DSpark increased total tokens generated per second across users by 51% and 52% for specified operating points on Flash and Pro respectively. Under higher guaranteed rates the reported gains reached 661% and 406%; DeepSeek notes that the older drafter nearly failed at those higher speeds, so these figures mark newly feasible operating points as well as speedups.
Background and why it matters
Speculative decoding was first described by authors at Google Research in 2022 and has become common in production serving. Early drafters were sequential (generating tokens one by one), exemplified by EAGLE‑3; parallel drafters like DFlash bypass the sequential bottleneck by drafting entire blocks at once. DSpark aims to combine the parallel drafter’s speed with the coherence of sequential approaches. DeepSeek replaced its simple sequential drafter two weeks after the DeepSeek‑V4 preview launched.
Practically, every token produced by a deployed model costs money and increases user wait time. Typical mitigations—using smaller or quantized models—sacrifice accuracy. DSpark reduces cost and latency without touching target model weights or degrading output quality, effectively making tokens cheaper and responses faster.
Takeaway
DSpark adapts verification effort to system load while combining a parallel backbone with a light sequential correction and calibrated confidence estimates. According to DeepSeek’s measurements, this yields substantial per‑user and aggregate speedups in production, and the company has released the DSpark checkpoints and code under a permissive MIT license for others to use.
