DeepSeek and Peking University Jointly Open-Source Speculative Decoding Acceleration Framework DSpark
DeepSeek and Peking University have jointly launched the DSpark large model inference acceleration framework, which has entered the DeepSeek-V4 series production system. With total throughput unchanged, DeepSeek-V4-Flash single-user generation speed increased by 60%–85%, and DeepSeek-V4-Pro increased by 57%–78%. DSpark adopts a semi-autoregressive architecture and confidence-scheduled verification mechanism, balancing draft generation speed and coherence, and can dynamically adjust verification length based on system load.
Just now, DeepSeek V4 updated DSpark, inference speed increased by 80%
Just now, DeepSeek V4 underwent an update.
It newly launched the Speculative Decoding framework DSpark, and simultaneously open-sourced the full-stack speculative decoding framework DeepSpec supporting this version.
DeepSeek-V4-Pro-DSpark is not a brand new architecture model, but rather introduces a speculative decoding module based on DeepSeek-V4-Pro. The focus of this update lies in engineering implementation, rather than iteration of the model capability itself.
DSpark has been deployed in real online traffic of DeepSeek-V4 (Flash and Pro), significantly accelerating the inference speed of large language models (LLMs).
Technical Report: "DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation"
Technical Report Link: https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf
The core intention of DSpark is to solve the latency and throughput bottlenecks faced by LLM inference in production environments (especially under high concurrency scenarios). In short, DSpark successfully combines high-throughput "parallel generation" with adaptive "load-aware verification".
Speculative decoding is a technology that accelerates large language model inference without changing the model output distribution. Its core idea is to introduce a lightweight "draft model" to pre-generate several candidate tokens, which are then batch-verified and accepted by the target model, thereby transforming serial token-by-token generation into parallel batch verification, significantly reducing end-to-end latency.
On this basis, DSpark's innovation lies in introducing a Semi-Autoregressive Generation architecture: it retains the high-throughput advantages of parallel draft models while adding lightweight serial modules to model the dependencies between tokens within a block, mitigating the acceptance rate decay problem that parallel draft models easily encounter at subsequent positions.
Additionally, there is hardware-aware confidence-scheduled verification: previous speculative decoding usually blindly sends all generated draft tokens for verification. Under high system load, these tail tokens with a very high probability of rejection severely waste valuable batch processing computing power. DSpark introduces a Confidence Head to evaluate the survival probability of each token. Combined with a hardware-aware prefix scheduler, the system can dynamically tailor the optimal verification length for each request based on real-time engine throughput characteristics, allocating computing power only to tokens with the highest expected return.
To implement this in real online infrastructure, DSpark's scheduler adopts an asynchronous mechanism to be compatible with Zero Overhead Scheduling (ZOS) and continuous CUDA graph replay. It uses historical predictions from the first two steps to decide the current dynamic truncation length, thereby hiding scheduling latency, avoiding GPU pipeline stalls, while ensuring complete lossless restoration of the target model output distribution.
In tests covering multiple fields such as mathematical reasoning, code generation, and daily conversation, DSpark significantly surpassed current state-of-the-art autoregressive models (Eagle3) and parallel draft models (DFlash). For example, on Qwen3 series (4B, 8B, 14B) target models, its average acceptance length increased by 26.7% to 30.9% compared to Eagle3, and by 16.3% to 18.4% compared to DFlash.
Compared to the previous generation's single Token production benchmark (MTP-1), while maintaining the same overall throughput, DSpark increased user generation speed by 60%-85% (Flash model) and 57%-78% (Pro model) respectively.
Along with DSpark, DeepSpec was also open-sourced, which is a full-stack code library for training and evaluating speculative decoding draft models. It is the "open-source infrastructure" carrying this solution and other cutting-edge algorithm implementations, including data preparation tools, draft model implementations, training code, and evaluation scripts.
DeepSpec divides the overall process into three stages: data preparation, training, and evaluation. The three stages need to be run in sequence, with the output of the previous stage serving as the input for the next stage.
In the data preparation stage, prompt data needs to be downloaded, answers regenerated using an inference engine for the target model, and a target cache constructed. It is worth noting that, taking the default Qwen/Qwen3-4B configuration as an example, the target cache volume can reach approximately 38 TB, so storage resources need to be fully evaluated before use.
The training stage can be started via bash scripts/train/train.sh. This script will call train.py and start a worker for each visible GPU. Users can select different algorithm and target model configurations in the config/ directory by specifying config_path. The project also supports adjusting training settings by overriding config_path, target_cache_dir, and using --opts to modify individual configuration fields.
In terms of hardware, DeepSpec's default configuration and scripts are oriented towards a single-node 8-card environment. If the number of GPUs is fewer, users need to correspondingly reduce the number of visible GPUs in CUDA_VISIBLE_DEVICES.
The evaluation stage is started via bash scripts/eval/eval.sh. The evaluation script will use the trained draft model checkpoint to measure acceptance on multiple speculative decoding benchmark tasks. The evaluation datasets currently listed in the project include GSM8K, MATH500, AIME25, HumanEval, MBPP, LiveCodeBench, MT-Bench, Alpaca, and Arena-Hard-v2, covering different task types such as mathematical reasoning, code generation, conversation capabilities, and comprehensive Q&A.
In terms of algorithms, DeepSpec currently has three built-in draft models: DSpark, DFlash, and Eagle3. Regarding target model series, the project currently supports Qwen3 and Gemma.
The open-sourcing of DeepSpec integrates speculative decoding, which was previously scattered within various research teams' engineering practices, into a reproducible, scalable standardized toolchain. For researchers and engineers hoping to accelerate inference for their own large models, this means they can directly train custom draft models on a mature framework, skipping a large amount of repetitive infrastructure setup work.
Reference Links:
https://github.com/deepseek-ai/DeepSpec/blob/main/DSpark_paper.pdf
https://github.com/deepseek-ai/DeepSpec
This article comes from the WeChat public account "Machine Heart" (ID: almosthuman2014), authors: Zinan, Yang Wen, published by 36Kr with authorization.
This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.