让光网络在坏掉前自救
训练大模型时，几万块芯片每一步都要同时交换数据，机架之间靠光网络连接。问题在于，从激光器到交换机这四层，每一层都只在故障发生后才反应，或者干脆按固定策略运行。而同一个机箱里的激光器共享散热和供电，会一起坏；在固定的备件比例下，这种相关性故障会让资源池的可用性有一个上限，池子再大也突破不了。这篇的做法是给每一层配一个预测控制器：先读出每个资源的下一个状态，在故障发生前就动手，四个部件各自带形式化保证——激光器池按覆盖上界定容、交换机调度不劣于最优的 11/9、微环谐振器提前一步控制、高风险模块优先保护。控制器不确定时就退回原有策略，不冒险。效果是：如果一整窗备件耗尽，有效吞吐（相对理想吞吐的保留比例）从 65.2% 升到 76.5%；只丢暗通道带宽时到 67.4%。折算下来，每兆瓦最多相当于 1.03–1.17 倍的训练量，同样的训练最多省 3.3–14.7% 的数据中心容量。它不是你明天能用上的东西，但如果你关心的是「大模型训练成本里，有多少是花在等网络恢复上」，这篇给出了一个可量化的答案。
📄 原文摘要(英文)
A frontier AI model is trained in an AI data center, on tens of thousands of accelerators joined by a dedicated network, and every training step ends with all of them exchanging data at once. Between racks, that network is optical, and each of its four layers, from lasers to switches, runs a fixed or after-the-fact policy. Lasers sharing an enclosure, cooling, and power can fail together. At a fixed spare ratio, correlated failure provably caps a pool's availability however large it grows. In this paper, we present a comprehensive predictive control plane framework, called Halley, that reads each resource's next state and acts before the failure occurs. It has four components, each carrying a formal guarantee: LightReserve that sizes the laser pool to provide a coverage bound, FoldFit that schedules the switch within 11/9 of the optimal, ThermaLead that holds a microring with one-step-ahead control, and TriageGuard that protects the highest-risk modules first. Halley gives each layer a predictive controller that defers to the deployed policy when unsure. Composed from simulations at that point, effective goodput, the share of ideal throughput kept, rises from 65.2\% to 76.5\% if a window that exhausts its spares is lost. If only the dark channels' bandwidth is lost, it reaches 67.4\%. Per megawatt, that is up to 1.03--1.17 times the training; for the same training, up to 3.3--14.7\% less data center capacity. Halley's fallback circuits are formally verified and are shown to achieve close timing on a commercial FPGA, and pass a 130~nm signoff.