Predictive Recording of LLMs 400% Faster

DSpark’s new DeepSeek module brings predictive innovation to DeepSeek-V4. It may look like a niche inference tweak, but in production it increased the generation speed of each user by 60 to 85 percent with no decrease in model quality.

What makes DSpark different is that it tackles two long-standing problems simultaneously, poor draft quality and wasted draft validation, where previous methods only addressed one. In this article, I will explain how it solves both and why that is important for production scale.

What is Imaginative Recording?

LLM generation is slow because each token requires a full forward pass through the model. Predictive coding accelerates this with a small draft model that predicts several future tokens at once, the target model and then validates them in a single pass.

If the draft model makes good predictions, several tokens can be generated from a single forward pass through the target model. If it makes negative predictions, it returns to its normal vineyard. The quality of the result is preserved because the target model verifies the predictions against its probability distribution.

The main problem is to create the right draft model:

• If it is sequential and accurate with long guesses, it cannot keep up with the target model and fails to generate more tokens before the target is exhausted.
• In this case, the delay ends up increasing based on the number of blocks being processed.

By making the draft model fast and consistent rather than sequential, predictions become less accurate in the latter part of the block. DSpark presents a solution that addresses both aspects at the same time.

Core Idea: Semi-Autoregressive Drafting

Here is a pattern for the prediction model: in the default context (ie, Eagle3), each generated token is appended to all previously generated tokens. Although this represents traditional machine learning, it is not efficient, as the model experiences a linear increase in latency when the number of tokens is generated.

In the same context (ie, DFlash), the model generates an entire block of tokens in one forward pass. This produces very fast output. However, each token is valued by dividing the others placed in the block. As you can imagine, the result from such a model can create a strange combination of words. Take “of” and “problem” for example: each makes a meaningful sentence (“of course” and “no problem”), but used together (“this problem”) they no longer make sense.

DSpark combines a highly parallel structure for speed (multiple independent processing paths) with a minimal sequence structure that adds spatial dependencies between tokens. Together, it is a highly consistent method with a thin layer of autoregression on top to correct inconsistencies between sequences.

The paper offers two sequence structures:

• The Markov algorithm uses only the previous token and the low-rank matrix, which does not get anything higher.
• The RNN header keeps a small repeating state for every block, giving you more context than the Markov header.

DeepSeek found that the Markov algorithm delivers all the benefits with much lower complexity, so it’s the one they put into production.

Starting with DeepSpec

DeepSeek has open sourced the training and testing code for their draft models as DeepSpec. This is the perfect repo for training any type of draft model, not just DSpark, but also DFlash and Eagle3. You can reproduce their comparison of those models using this repo.

To install dependencies and compile this repo, see the README files included in the repo.

git clone 
cd DeepSpec
python -m pip install -r requirements.txt 

This includes training and testing models with DeepSpec. However, you will still need to prepare your data separately by using a method to get results from the target model. For more information on how to do that, contact scripts/data/README.md in this repo.

Hands-On: Training and Testing a Draft Model

There are three stages in the DeepSpec workflow: preparing the data, training your model from scratch, and testing it. The output of one stage becomes the input of the next stage.

Step 1: Selecting Config

You can find the configuration in the config/folder (there is one file for all algorithms and target models).

ls config/dspark/
# dspark_qwen3_4b.py  dspark_qwen3_8b.py  dspark_gemma4_12b.py 

Each configuration file specifies the Target Model, Block Size, and which sequence header to use. If you want your setup to match the smallest benchmark described in the paper, you’ll want to use dspark_qwen3_4b.py configuration file.

Step 2: Training the model

To start training, you will run the following command:

bash scripts/train/train.sh --opts config_path=config/dspark/dspark_qwen3_4b.py 

The script may create a worker for each GPU on your system. Test files will be saved in it ~/checkpoints///step_*. If you are using only one training node, you will need to set it up CUDA_VISIBLE_DEVICES change to match the number of GPUs you have.

Within the training process itself, we prepare for three types of losses at the same time:

• cross-entropy term (for predicting the next token correctly),
• distribution matching term (directly related to the “acceptance rate” of the content produced),
• “loss of confidence”

This last one is important, as it allows us to use the planning strategy described in the next section.

Step 3: Testing

bash scripts/eval/eval.sh  
  --target_name_or_path Qwen/Qwen3-4B  
  --draft_name_or_path 

~/checkpoints/deepspec/dspark_block8_qwen3_4b/step_latest

Verification happens in one pass, so measure how many tokens are accepted for three types of activity: math, code, and chat. More tokens accepted means fewer wasted passes to the target model.

Test Results

The statistics presented by DeepSeek were truly remarkable. DSpark exceeded the acceptable height of Eagle3 by about 27-31%. DSpark output exceeded DFlash by 16-18%. Both improvements remained consistent across all targets for Qwen3-4B, 8B, and 14B. Furthermore, they did the same for Gemma4-12B as well, which shows that there is something to Gemma’s effects and not just Qwen’s quirk.

The result of the separate families helps explain why the DeepSeek post has the titles of both Gemma and Qwen listed. This should be considered a better indicator than a comparison with only one model. Architecture-specific strategies often break down when tested on another model branch.

Gotchas and Human Fascinations

Here are some very important pieces of information, regardless of the form in which the information is presented:

• Dialog Confirm Code Mismatch: The dialog has more valid tokens (meaning it has a lower confidence level) than the code, so the confidence decreases quickly and the editing will be very slow.
• Static Thresholds Are Not Dynamic Programming: The static cutoff is last year’s technology, and the cutoff does not consider how busy your system is, DSpark will recalculate the dynamic cutoff for each batch.
• Causality is non-negotiable: Because you can’t see into the future, the programmer can’t check the token before it can confirm that the token is valid. This is often controlled offline using the two-step confidence prediction process that was in place at the end of V2.
• At the extremes the standard percentages are very misleading: For example, the 661% multiplier for MTP-1@V4-Flash is under synthetic conditions, the metric does not reflect the manufacturer’s real world output so don’t use the multiplier as an expected value instead use the 60-85% simulated output.
• You cannot reimburse the cost of writing: Even if your query is not accepted and you still pay the full writing fee at the time of the query, even if the system cancels the verification after editing.

The conclusion

DSpark is a stark reminder that theoretical speedups can come from many places. Not every profit requires a bigger model or better hardware; sometimes it comes from admitting that the draft may be wrong and allowing the editor to work around that acceptance wisely.

If you use virtual logging under various application loads, the concept works even if your architecture is not the same as DeepSeek. The premise is simple: only validate what has a positive expected value.

And if you’re wondering how Markov’s head fits into the full attention of the draft block, that’s the next rabbit hole to chase. You can test it yourself, because the DeepSpec repo has everything you need.

Frequently Asked Questions