AI 世界模型:只会“常见”不会“反常”
AI 生成图片和视频越来越逼真,但新研究揭了底:它们只会模仿“常见”的物理互动,遇到“反常”场景就露馅。研究者造了个测试集,分三档:常规(锤子钉钉子)、非常规(用砖头钉钉子)、不可能(用棉花钉钉子)。结果模型在常规上表现不错,但非常规和不可能场景下性能断崖下跌——它们根本没学会物理原理,只是记住了常见画面的表面模式。比如,让模型生成“用棉花钉钉子”,它可能把棉花画成锤子形状,或者干脆不改变钉子状态。视频模型更惨,连时间连续性都保不住。这不是你明天能用上的工具,但它提醒你:别被 AI 的“逼真”骗了,它离真正理解世界还差得远。
📄 原文摘要(英文)
Physical interactions follow a long-tailed distribution: a set of common and regular interactions dominates human experience and visual data, while a broad spectrum of rare and irregular interactions remains underrepresented. Although recent visual world models, including image and video generation models, achieve impressive realism on existing benchmarks, they primarily focus on simulating common physical interactions. This raises a central question: Do current visual world models internalize and generalize physical principles? In this work, we introduce Tailor-Bench, a benchmark that challenges world models to simulate irregular physical interactions. To enable systematic evaluation, we design three scenario modes that progressively challenge model reasoning: Regular scenarios reflect common tool-task pairs, Unconventional scenarios replace conventional tools with attribute-compatible substitutes to test affordance generalization, and Impossible scenarios introduce attribute-violating tools to probe constraint awareness. Additionally, we design two complementary settings under a unified evaluation protocol: predictive generation requires inferring outcomes without guidance, while descriptive generation specifies the target outcome for faithful realization. Our experimental results reveal a clear long-tail gap in physical world modeling: performance degrades from Regular to Unconventional and Impossible scenarios, indicating limited generalization beyond common interactions. Failure analysis further shows that models rely on superficial visual patterns: image models fail to realize correct state changes, while video models further suffer from temporal inconsistencies.