AI推理提速新招：只验证靠谱的预测
大模型生成回答时，通常是一个词一个词往外蹦，很慢。业界有个加速技巧：让模型先快速猜一串词，再整体检查。但猜得越长，后面越容易错，白费算力。DSpark 做了两件事：一是让猜词时保留一点前后依赖，避免后半段乱猜；二是根据每个请求的“存活概率”动态决定检查多长，不靠谱的词直接跳过。在 DeepSeek-V4 系统中，用户生成速度提升了60%到85%，而且在高并发下不会卡死。这不是你明天能用的工具，但它是大模型服务商提升响应速度的新方向。
📄 原文摘要(英文)
Speculative decoding accelerates Large Language Model (LLM) inference by decoupling draft generation from target verification. While recent parallel drafters efficiently propose long token sequences in a single forward pass, they suffer from rapid acceptance decay due to a lack of inter-token dependencies. Furthermore, indiscriminately verifying these extended blocks wastes critical batch capacity on tokens with high rejection risks, severely degrading throughput in high-concurrency serving systems. We introduce DSpark, a speculative decoding framework that unifies high-throughput parallel generation with adaptive, load-aware verification. To maintain draft quality, DSpark utilizes a semi-autoregressive architecture, coupling a parallel backbone with a lightweight sequential module, to introduce intra-block dependency modeling and mitigate suffix decay. To optimize system efficiency, DSpark employs confidence-scheduled verification, dynamically tailoring the verification length for each request based on estimated prefix survival probabilities and engine-specific throughput profiles. On offline benchmarks across diverse domains, DSpark substantially improves the accepted length over state-of-the-art autoregressive and parallel drafters. When deployed within the DeepSeek-V4 serving system under live user traffic, DSpark successfully mitigates verification waste. Compared to the established production baseline (MTP-1), DSpark accelerates per-user generation speeds by 60 to 85 percent at matched throughput levels. More importantly, by preventing severe throughput degradation under strict interactivity constraints, it enables performance tiers that were previously unattainable, shifting the Pareto frontier of our serving system.