让AI视频生成连续跑24小时不崩的秘诀:给模型分两个大脑
现在的AI视频生成,每生成一帧都要同时干两件事:把当前帧画清楚,再把这一帧存成记忆供后续参考。研究者发现这两件事共用一套参数,但它们的训练信号方向经常相反——一个想让画面更清晰,一个想让前后更连贯,结果互相拖后腿。SGF+的做法很简单:把这两个任务拆开,各配一套独立参数,让它们通过注意力机制协作。训练时只用了5秒的视频片段,但模型能连续生成24小时不崩,画面质量和长期一致性都超过现有方法。这说明给模型的不同职责分配独立参数,是提升视频生成质量的一个有效设计原则。
📄 原文摘要(英文)
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typically share parameters, and we find that their gradients exhibit distinct patterns and systematic negative alignment, hindering the joint optimization of visual quality and temporal consistency. We introduce Self Gradient Forcing Plus (SGF+), which assigns separate parameters to context writing and denoising while preserving their interaction through causal attention. Both roles are jointly optimized using the original generation objective without auxiliary losses, with context writing supervised through its contribution to future predictions. This simple change improves visual quality and long-horizon consistency over the evaluated baselines in both framewise and chunkwise generation, without additional video training data or a longer training horizon. Trained on only 5s rollouts, SGF+ supports continuous generation for up to 24 hours without long-video fine-tuning. These results highlight role-specific parameterization as an effective design principle for high-quality autoregressive video generation and native long-horizon extrapolation.