把乱糟糟的室内视频变成可编辑的3D场景
现在的机器人仿真想要一个「真实房间的复制品」，得先有干净的数据：每个物体拍得清清楚楚、没有遮挡。但真实房间永远是乱的——沙发挡着茶几，花瓶只露半边。Lucida 的思路是：别在开头要求完美，把精度留到最后。它先把视频拆成一张「场景图」，每个物体挂上自己所有角度的证据；再为每个物体生成完整模型，哪怕只看到一半也能补全；最后用一个会「自己看、自己调」的 AI 来摆放物体，像在建模软件里拖拽 gizmo 一样，直到对齐为止。结果：物体检测精度提升 69%，位姿估计从 57.8% 涨到 83.4%，场景重建 F-Score 从 0.794 到 0.924。这不是你明天能用上的工具，但它意味着机器人训练场可以更快地从真实房间自动生成——以后给机器人「看」一个家，它就能自己搭一个能拆开玩的副本。
📄 原文摘要(英文)
Composable scene modeling aims to recover a real indoor scene as complete, editable object assets arranged as observed, giving robot simulation and embodied AI a simulation-ready replica of the real environment whose objects can be manipulated individually. Existing pipelines decompose the task into three steps---parse the observations into instances, generate an asset for each, and place each asset back---but every step presumes an input that a cluttered capture rarely provides: accurate instance geometry, unoccluded views, and assets that accurately match the observations. We propose Lucida, which keeps this order but redistributes the requirements, so each step consumes only what a real capture reliably provides and precision is reached at the end of the pipeline rather than demanded at its start. Lucida parses the video into a scene graph whose nodes carry per-instance multi-view evidence, generates a complete asset for each instance from its evidence, and places assets with GizmoAct, a VLM policy that casts placement as multi-turn GUI interaction, manipulating the object's gizmo in a closed loop and deciding itself when alignment is reached. Across scene-level 3D object detection, object pose estimation, and scene reconstruction, Lucida improves mAP over Boxer by 69% on R2S-Scene, raises [email protected] from 57.8% to 83.4% on CA-1M, and increases scene F-Score from 0.794 for SAM3D to 0.924.