AI 自己给自己提速，从 4 倍到 11 倍
大模型回答一个字，就要把整个模型读一遍，所以慢。业界用「小模型先猜、大模型只验」的投机解码来提速，但官方给的配置远没榨干潜力。这篇让 AI 自己当工程师：先算出当前瓶颈在哪，再让多个智能体并行试各种优化，试对了就留下、试错了就扔掉，还专门防「作弊式提速」。结果不训练、不掉精度，单次请求提速从 4.46 倍拉到 10.87 倍，批量场景从 1.94 倍到 3.93 倍。它不是你明天能用上的，但说明「AI 调优自己」这条路已经跑通——以后模型厂商可能不用再靠人肉调参，直接让模型自己找最优解。
📄 原文摘要(英文)
Speculative decoding accelerates memory-bound LLM decoding with a lightweight drafter, and parallel drafters extend this speedup by drafting a whole block in one forward pass. However, it remains unclear whether their released systems (stock) attain the full speedup that parallel drafting permits. We identify three challenges. First, with one request, each step accepts few tokens but reads every target weight. Second, with batching, per-request verification work roughly halves the speedup over autoregressive (AR) decoding. Third, speculative decoding is deployment-specific, and no universal optimization fits every serving regime, drafter, target model and GPU. To address these challenges, we present SpecPilot, an autonomous agentic loop that discovers algorithms and systems optimizations for each deployment. In each round, a cost model locates the largest gaps between the current system and its ceiling. Parallel agents explore directions and integrate the successful ones. Token forcing checks every output token against stock to detect reward hacking. A structured memory carries validated mechanisms to new rounds and deployments. SpecPilot discovered a general lossless 12-bit weight format, tree verification for hybrid recurrent targets, two-level verification and three batching mechanisms. We evaluate it across four target models, three drafters, two GPU platforms and five benchmarks. Without training or quality loss, it raises the speedup over AR from stock's 4.46times to 10.87times with one request, and from 1.94times to 3.93times with batching.