AI Pulse
📄 论文解读

机器人单会想、单会做,一合起来就崩

机器人领域有个尴尬的现状:有的模型很会「想」,有的很会「做」,但让同一个机器人边想边做,成功率从五成掉到一成二。这篇论文造了一个 180 种配置的模拟测试场,把任务拆成三种:只看历史做决策、只看动作执行、以及两者闭环结合。结果最强的系统单独想能对 52.7%,单独做能对 39%,一旦合起来只剩 12%。问题不在它缺哪块能力,而在它没法把两块接上——想的时候顾不上手,做的时候忘了刚才看到什么。这不是你明天能用上的东西,但它指出了一个真问题:我们可能一直在分别训练机器人的大脑和手脚,却没人好好教它们怎么配合。

📄 原文摘要(英文)

Embodied intelligence aims to let robots perform complex physical tasks with human-like adaptability and reliability, which requires three complementary capabilities: history-based cognitive reasoning, precise and timely physical execution, and the composition of reasoning and execution through environmental feedback. Yet existing benchmarks either evaluate reasoning and execution within the same tasks or assess them separately without their composition, so they reveal little about whether a failure comes from reasoning, execution, or their composition. We introduce Mind2Act World, a diagnostic benchmark of 30 simulated manipulation tasks from five real-world application domains. Each task has three difficulty levels and two scene settings (Clean and Random), yielding 180 configurations. The tasks form three views: Mind tests decisions based on past observations, Act tests precise and timely execution, and Mind2Act combines both in closed-loop tasks where action outcomes guide later decisions. Comparing the three views shows whether a system lacks a capability or fails to compose capabilities it already has. A diagnosis harness further attributes the failed episodes of three representative systems to Mind, Act, or their composition and summarizes each system in a profile card. Our evaluation of multimodal agents, vision-language-action models (VLAs), and world-action models (WAMs) shows that competence in a single view does not carry over to Mind2Act tasks. Even the strongest agent succeeds in 52.7% of Mind and 39.0% of Act episodes but in only 12.0% of Mind2Act episodes. The strongest agent retains task state but struggles with sustained contact, whereas end-to-end policies execute practiced motions but retain little task state. These findings point to composing reasoning and execution through environmental feedback as the main open problem.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新