让AI学会“看东西动起来”
现在的视觉AI能回答“杯子在桌子左边”这种静态问题，但一旦物体动起来、视角变了，它就懵了。这篇论文给AI设计了一套“物理课”:先让它看物体自己移动(被动)，再让它看自己移动时世界怎么变(主动)，最后让它跟着一条完整轨迹走，把一连串变化串起来。训练数据来自模拟和真实世界的交互轨迹，分三级递进，效果在多个模型和基准上都有提升。它不是你明天能用上的东西，但它指向一个关键方向：AI要真正理解物理世界，不能只看照片，得学会看“变化”。
📄 原文摘要(英文)
Spatial reasoning is essential for vision-language models (VLMs) to understand and act in the physical world. Reasoning in dynamic environments requires VLMs to perceive local state transitions caused by object motion and viewpoint changes and integrate them over long trajectories to maintain an updated spatial state, yet existing VLMs remain limited in both capabilities. Current spatial training primarily focuses on static questions about object attributes and spatial relations, providing limited direct supervision for state transitions; in contrast, interaction trajectories naturally connect a preceding observation, an action, and a subsequent observation, offering direct supervision for local state transitions, while complete trajectories reveal dependencies among consecutive transitions. We therefore introduce Spatial-Interactor, a framework that trains VLMs to model physical-world state transitions through interaction, organizing this learning process into a three-level curriculum covering L1 passive world-state transitions, L2 active self-state transitions, and L3 long-horizon interaction trajectories. Accordingly, we construct the Learning from Spatial Interaction dataset (LSI-108K) from simulated and real interaction trajectories, with tasks aligned with the objective of each level. Our two-stage training strategy applies Supervised Fine-Tuning (SFT) to L1 and L2 for local transition modeling, and On-Policy Distillation (OPD) then uses privileged self-distillation: a teacher branch given segment-level transition descriptions supervises the student's on-policy CoT, helping the student learn to integrate consecutive transitions over L3 long trajectories. Experiments across multiple VLMs and spatial benchmarks show consistent gains in local transition modeling and long-horizon integration.