让机器人装箱像人一样边看边想
机器人装箱一直靠死板的几何规则或反复试错，换个箱子布局就抓瞎。PackLab 把多模态大模型请进这个场景：它一边看箱子里还剩多少空间、物体长什么样，一边决定下一个该放谁、放哪，全程闭环，而不是预先算好一套动作。在多种物体和容器组合下，它平均表现超过了传统启发式算法、强化学习方法和通用大模型。这不是你明天能用的技术，但它指向一个趋势：大模型正在从聊天走向需要长期判断的物理操作。
📄 原文摘要(英文)
Robotic bin packing requires long-horizon sequential decision-making, as each object placement affects the available space for subsequent packing. Existing methods primarily rely on hand-crafted geometric heuristics that optimize predefined objectives or reinforcement learning policies learned through trial and error over predefined training configurations. Despite recent advances in multimodal large language models (MLLMs) for this task, their potential for closed-loop sequential decisions across heterogeneous packing configurations remains underexplored. To address this gap, we introduce PackLab, a comprehensive framework for developing, training, and evaluating MLLMs for closed-loop robotic bin packing. PackLab-Suite provides a physics-based simulation platform for scalable generation of diverse training packing trajectories and evaluation of their physical outcomes. PackLab-VLM is a packing-specialized MLLM that understands the evolving object and container states to jointly select objects and predict placements in a closed-loop manner. PackLab-Bench provides standardized packing scenarios at multiple difficulty levels for systematic evaluation. Extensive experiments demonstrate that, on average, PackLab-VLM outperforms conventional packing heuristics, traditional reinforcement learning methods, and general-purpose MLLMs across object sets and container configurations, highlighting the potential of MLLMs for long-horizon robotic packing. The code, model, dataset, and benchmark are available at https://github.com/Correr-Zhou/PackLab .