异步流水线并行训练：延迟不是障碍，优化器才是
训练大模型时，GPU常因同步流水线中的气泡而闲置。异步流水线能消除气泡，但梯度延迟被认为会导致训练不稳定。这篇研究挑战了这一假设：延迟的破坏性取决于优化器选择，而非固有缺陷。AdamW在延迟下性能严重下降，而新优化器Muon则表现出强鲁棒性。研究者还引入了一种与优化器无关的误差反馈修正，进一步缓解延迟影响。在100亿参数模型上的实验表明，异步流水线可达到与同步训练几乎相同的性能。这不是你明天能直接用的技术，但它揭示了异步并行在大规模训练中的潜力，可能改变未来训练基础设施的设计。
📄 原文摘要(英文)
Modern large-scale LLM pretraining benefits from utilizing Pipeline Parallelism; however, synchronous implementations leave GPUs idle during pipeline bubbles, wasting computational resources. Asynchronous Pipeline Parallelism eliminates these bubbles, maximizing throughput at the cost of gradient staleness. Among asynchronous schedules, PipeDream-2BW is particularly appealing: unlike the original PipeDream schedule, it ensures a constant one-step gradient delay regardless of pipeline depth. However, its adoption remains limited due to the common belief that optimizing under staleness is fundamentally unstable. In this work, we challenge this assumption, demonstrating that degradation under one-step delay depends strongly on optimizer choice rather than being an intrinsic limitation. We provide the first comprehensive empirical analysis showing that while AdamW, the predominant optimizer at the time when PipeDream-2BW was introduced, indeed suffers from severe degradation, recent methods like Muon exhibit strong robustness under a one-step delay. We introduce an optimizer-agnostic Error Feedback-inspired correction to further mitigate delay effects. We provide supporting theoretical analysis demonstrating convergence for Muon with and without this correction. Extensive evaluation on models up to 10B parameters confirms that our strategies bridge the performance gap with synchronous training, highlighting the practical potential of asynchronous pipeline parallelism at scale.