AI 当管家,盘子掉下来它接得住吗
让大模型当机器人的大脑,面对突然掉落的盘子、飞来的刀,它能不能像人一样瞬间做出反应?现有测试只让 AI 看视频答题,或者做导航、整理这类慢任务,没人测过它能不能把物理直觉变成保命动作。这个新基准把 AI 放进模拟人体里,扔给它 17 类、上千个家庭突发险情,甚至故意用「泡沫做的铁砧、钢铁做的苹果」这种外表和物理属性矛盾的物体来骗它。结果七个主流模型平均每三次就搞砸一次:有的不看眼前场景、凭固定习惯乱动,有的信外观不信运动轨迹,就算选对了动作,拦截点也偏出好几米——而且模型越大,这些毛病一点没见好。它不是你明天能用上的东西,但它是给「AI 管家」上岗前补的一堂安全课。
📄 原文摘要(英文)
Reacting to sudden physical hazards (catching a slipping plate, dodging a falling knife) is both a meaningful test of embodied intelligence and a hard requirement for deploying multimodal large language models (MLLMs) as the decision coreof household robots. Existing evaluations, however, probe intuitive physics passively through question answering over videos, or target deliberate, long-horizon tasks such as navigation and rearrangement; none measure whether a model can turn physical understanding into immediate, safety-critical action. We introduce ReactHuman, the first physics-grounded benchmark for human-like reactive decision-making, in which the evaluated MLLM acts as the brain of a simulated humanoid facing sudden household hazards; it spans 17 event families and over 1,000 bit-for-bit reproducible scenes with exact, annotation-free ground truth derived from 240 Hz rigid-body simulation, including adversarial objects whose appearance contradicts their physics (a foam anvil, a steel apple). We further design a five-metric suite that scores each reaction along three axes: reasonable, safe, and physically grounded. We physically execute every committed plan so that decisions have observable consequences. With this harness we evaluate seven representative MLLMs. Results show that reactive safety is far from solved: models mishandle roughly one hazard in three, act from fixed dispositions rather than the observed scene, trust appearance over motion, and miss interception points at meter scale even when the chosen action is correct; none of these failures shrink with model scale. ReactHuman thus offers both a fine-grained diagnosis and a scalable training signal toward physically grounded, safety-aware embodied agents. The benchmark can be found here: https://huggingface.co/datasets/Alan123/reacthuman-benchmark-scaled