Dspark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
DeepSeek is a small Chinese lab with limited resources: its team is roughly 20 times smaller than OpenAI's and, because of export restrictions, it does not have access to Nvidia's most advanced GPUs.
For exactly this reason it cannot rely on ever bigger models or unlimited compute alone. It has to find smarter approaches.
And this is where things get interesting: DeepSeek is focusing on efficiency, meaning how to get more out of the models that already exist.
In its latest paper, dedicated to DSpark, the team presents a system that speeds up model generation by up to 85% without compromising output quality.
But to understand why this matters, we need to start from how language models work today.
How models generate: one token at a time
When we use ChatGPT, Claude or DeepSeek, we often get the impression that the model is "thinking". In reality, during the response phase something far more mechanical happens: the model generates text one token at a time.
- First it picks the next token.
- Then it looks at everything it has written so far.
- Then it picks the next token.
- And so on.
This is one of the reasons long responses take time: the model does not generate the whole paragraph in one shot. It has to build it up step by step.
Speculative decoding: the boss and the intern
A technique to speed up this process has existed for a while: speculative decoding.
The idea is simple.
Imagine the large model as a very capable but slow boss. It always has the final say, so quality stays high, but it writes one word at a time.
To speed things up, you pair it with a much faster intern. The intern tries to write a few words ahead. The boss no longer has to invent everything from scratch, it just reviews the draft.
If the draft is correct, it accepts several words at once. If it finds a mistake, it discards everything from that point on and starts again.
The advantage is that the large model still makes the final decision, so output quality is not sacrificed. But if the intern guesses right often, generation becomes much faster.
The problem: drafts that are accurate or fast, never both
The problem is that these "interns" have two opposite flaws.
Some are accurate but slow, because they also generate one token at a time. Others are extremely fast, because they try to generate several tokens in parallel, but they become less reliable as the draft grows longer.
DSpark's solution: Markov head and confidence head
With DSpark, DeepSeek tries to solve exactly this problem.
The solution is elegant: it uses a fast drafter, but adds two small control mechanisms.
The first is a Markov head: a lightweight component that looks at the token just generated and helps pick the next one better. It does not reconstruct the entire history of the text, but it gives the system a minimum of local continuity.
It is a bit like saying that if you have just written "some", the next word is probably "times", not a random word like "cat".
The second is a confidence head: for every proposed token, the system estimates how likely it is that the large model will accept it.
If confidence is high, it keeps generating a longer draft. If confidence drops, it stops before wasting compute on words that would probably be discarded.
Accelerating only when it pays off
This is the most interesting part: DSpark does not always try to run at maximum speed. It decides dynamically when it makes sense to accelerate and when it is better to be more cautious.
On more deterministic tasks, like code or math, it can afford longer drafts. On more open-ended tasks, like creative writing, it tends to stop earlier.
On top of that, the system also takes hardware load into account: when there are few requests it can push harder; when the server is under pressure, it shortens the drafts to avoid saturating the GPU.
The result is a system that not only speeds up a single user's response, but improves the overall efficiency of serving.
Why it matters: not just bigger models
And this is the most important point of the paper: progress in LLMs does not come from bigger models alone.
It also comes from smarter ways of running them.
DSpark points to a very concrete direction: more speed, less wasted compute, better hardware utilization and no need to compromise the quality of the main model.
In a world where agents take on ever longer tasks, this is not just a technical optimization.
It is a fundamental part of the user experience.