机器人评测：别只看画面美，要看它能不能演完一整集
评测机器人AI，比评测大语言模型难得多——你不能在云端跑个分，得让真机器人在真世界里动，慢、贵、还得人盯着。所以研究者想用「世界模型」来模拟评测：让AI在虚拟环境里跑一遍，看它行不行。但问题来了：什么样的世界模型才靠谱？这篇论文用7个视频模型、32万次模拟跑动，加上真实机器人数据，给出了反直觉的答案——画面好不好看（短期视觉真实感）不重要，重要的是模型能不能在长任务里保持动作一致、不跑偏。而且，预训练数据不是越多越好，得平衡通用知识和机器人特有的可控性。他们据此造了个专用评测模型GigaWorld-1，并开源了全部代码和数据。这不是你明天能用的工具，但如果你关心机器人什么时候能真正走进生活，这篇告诉你：评测标准本身，正在被重新定义。
📄 原文摘要(英文)
Evaluating embodied robot foundation models remains a critical bottleneck; unlike large language models efficiently assessed via digital benchmarks, robotic policies require slow, costly real-world rollouts limited by hardware and human supervision, which has driven interest in world models as surrogate policy evaluators, yet the key properties that make a world model reliable for policy assessment remain poorly understood. This work presents a systematic study of world models for robotic policy evaluation and introduces WMBench, a benchmark constructed from real-robot teleoperation data and matched policy rollouts covering diverse manipulation tasks to enable controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics. Using WMBench, we analyze 7 video world models, 4 action representation schemes, and over 324,000 simulated policy rollouts paired with real robot executions, further enriching our analysis with large-scale community submissions from the CVPR 2026 GigaBrain Challenge, curated synthetic trajectories, and a training videos spanning more than 12,000 hours. Our experiments deliver three core insights: evaluator quality is dominated by long-horizon, action-faithful rollout consistency rather than short-term visual realism; pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability; and architectural choices including action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior. Drawing on these results, we derive a practical design roadmap and realize it in GigaWorld-1, a world model specially optimized for policy evaluation, and we fully release our code, models, datasets, and toolkits to advance scalable evaluation research for embodied foundation models.