AI生成视频终于有立体声了，声音会跟着画面动
现在的AI生成视频，声音是单声道：画面里人从左走到右，声音却钉在正中间，沉浸感直接归零。这篇论文做了个叫StereoBind的框架，专门解决「声音位置要跟着画面里的声源动」这件事。做法不玄：先让模型认出画面里哪个物体在发声，再记录它每一帧的绝对位置，最后把相对位移也编码进去，三路信息合起来生成立体声。他们还造了2.9万条带运动轨迹的立体声视频数据来训练和评测，结果声音和画面的空间对齐明显好于现有模型。它不是你明天就能用的工具，但AR/VR和游戏里那种「声音从你左边飘到右边」的沉浸感，是这么一步步补上的。
📄 原文摘要(英文)
Recent joint video-audio generation models have achieved strong semantic correspondence and temporal synchronization. However, applications such as AR/VR and interactive gaming further require stereo audio to provide an immersive sense, which remains largely overlooked. Effective stereo audio requires the perceived sound location to evolve consistently with the motion of its corresponding visual source. We refer to this property as Dynamic Spatial Correspondence and propose StereoBind, a framework that binds visual source motion to stereo sound generation. StereoBind uses motion tracks to coordinate visual motion and stereo audio through three complementary mechanisms. Visual Motion Binding establishes source-aware audiovisual correspondence, the Spatial Track Encoder captures absolute source positions, and Residual Track RoPE models relative motion. For supervision and evaluation, we construct StereoWorld-29K, a large-scale stereo audio-video dataset with paired motion tracks, and StereoWorldBench for measuring audiovisual spatial consistency. Experiments show that StereoBind substantially improves spatial alignment in stereo audio generation over existing models while preserving overall audiovisual quality.