让AI看视频时也懂物理：预测每个点的3D轨迹
现在的视频生成模型能编出像样的画面，但画面里的物体常常违反物理——杯子掉下去会穿桌，人转身时脸会糊。这篇论文给AI加了一双「物理眼」:不只预测未来画面，还同时预测画面里每个物理点在3D空间中的移动轨迹，用多台摄像机的视角交叉验证，让AI真正理解「东西在动」而不是「像素在变」。结果：视频更逼真，3D运动更准确，而且这套3D理解直接迁移到机器人控制上，机械臂的操作成功率也提高了。它不是你明天就能用的工具，但这是视频生成从「画得像」走向「懂物理」的关键一步。
📄 原文摘要(英文)
Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. We further extend VPM to robot control by adding robot actions as a token modality in the generator. Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.