AI重建3D场景,终于学会“边看边改”了
过去AI重建3D场景像拍照:一次性拍完,想加个角度就得重拍。这篇让AI像人一样“边看边改”——先粗略搭个框架,每来一张新照片就顺手修正全局,不用重算旧图。它还能记住已处理的部分,只更新新信息,效率高得多。这不是你明天能用的工具,但它意味着3D扫描、AR导航这类技术,未来可能不再需要“重新建模”的漫长等待。
📄 原文摘要(英文)
Classical structure-from-motion reconstruction proceeds iteratively, refining camera poses and scene geometry from a current estimate. This allows maps to easily expand without recomputing already computed views, or geometry estimates to improve as more observations are provided. Most geometric foundation models, however, predict entire geometry in a single forward pass or update a recurrent state as frames arrive sequentially, making this expansion and refinement non-trivial. We introduce Solv3R, a looped transformer that learns to continue solving multi-view geometry from its current state. Importantly, its update operator is independent of the iteration count, allowing previously refined and newly introduced views to be jointly updated regardless of how ``solved'' they are. We train the operator to preserve accurate reconstructions under continued iteration and correct inaccurate states, enabling refinement of fixed image collections and integration of new observations. Representations of solved views can also be cached once and reused as geometric context across iterations, enabling compact and efficient key-value caching. We analyze the solver dynamics and compare it with the state-of-the-art, demonstrating that a single learned solver supports offline reconstruction, incremental reconstruction, relocalization, and streaming without task-specific training or modules. The code and models will be made publicly available at https://ubc-vision.github.io/solv3r/.