AI推理提速新招：边猜边验，不浪费算力
大模型回答慢，通常是因为一个字一个字地生成。DSpark 让模型先并行猜出一段话，再快速验证，但猜得太长容易错，验证又浪费算力。它用半自回归结构在并行中保留少量顺序依赖，让猜出的内容更靠谱；同时根据每个请求的置信度动态调整验证长度，避免在容易错的地方浪费资源。在 DeepSeek-V4 系统中，用户生成速度提升了60%到85%，且在高并发下不会严重降速。这不是你明天能直接用的工具，但它是大模型服务商提升响应速度的关键技术，未来你用的AI助手可能会更快。
📄 原文摘要(英文)
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.