视频AI的下一个战场:从看懂画面到理解世界
视频AI正在换赛道。过去十年,它忙着把视频看懂、把画面生成得更逼真;这篇综述指出,真正的分水岭是「世界模型」——把视频当成世界的一个不完整投影,AI的任务从识别画面升级为推断画面背后那个持续演化的世界状态。研究者把现有工作归成四类:从视频反推世界属性(读)、按条件生成视频(写)、把理解和生成统一进同一套机制(共享)、以及根据动作和不断到来的新观察预测世界下一步(交互)。趋势很清楚:评价标准从「画面像不像」转向「世界状态连不连贯、物理和因果对不对、能不能支撑下游决策」。它不是你明天能用上的东西,但如果你关心AI离「真正理解现实」还有多远,这是目前最清晰的路线图。
📄 原文摘要(英文)
Video foundation models have advanced rapidly in understanding and generating visual sequences, while video world models seek to capture the underlying dynamics of the world they depict. Despite their different objectives, both concern the relationship between video observations and an evolving world. This survey adopts world modeling as a unifying lens for examining these developments. Viewing an observed video as a partial, lossy, and temporally ordered projection of a latent temporal world, we organize the literature around four complementary operations: reading infers world properties from observed video; writing synthesizes video from specified or learned world conditions; sharing integrates understanding and generation through common modeling mechanisms; and interacting predicts and updates world states under actions and evolving observations. This perspective distinguishes temporal coherence in observed or generated sequences from consistency in the underlying world state. Across tasks, representations, models, datasets, and benchmarks, we trace a shift in emphasis from clip-level recognition and perceptual fidelity toward persistent state, controllable evolution, physical and causal validity, and evaluation grounded in interaction and downstream decisions, and we distill the open challenges that follow.