Tencent Open-Sources AngelSpec: A Unified Training Framework for MTP and Block-Parallel Speculative Decoding on Hy3 Models
Tencent open-sourced AngelSpec, a PyTorch-native framework for training speculative decoding draft models across six architectures, featuring a new block-diffusion drafter called DFly.
Tencent released AngelSpec, an open-source tool designed to streamline the training of draft models used in speculative decoding. The framework supports six different model architectures and is built natively on PyTorch.
A key component is DFly, a block-diffusion drafter that utilizes hybrid target conditioning. It also includes a hidden-correction autoregressive head to improve accuracy during the decoding process.
Speculative decoding aims to accelerate LLM inference by using smaller draft models to predict tokens verified by larger target models. By providing a unified training framework, Tencent hopes to lower the barrier for developers optimizing inference efficiency.
This release aligns with industry efforts to reduce latency and computational costs associated with large language model deployment. Open-sourcing such tools typically encourages broader adoption and community contribution to inference optimization techniques.
This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.