生成室内场景,终于又快又多样了
现在的 AI 生成室内 3D 场景,要么慢得离谱——一个场景要反复摆放、调整物体,要么快但结果经常穿模、家具悬空。这篇把两条路拼起来:先用图片先验快速搭出初稿,再用视觉语言模型做局部微调,把场景拆成独立小单元分别进化,最后组合。结果:质量不输最慢但最精细的方案,速度快了 24 倍,物理错误大幅减少,人类评分最高。它不是你明天就能用的工具,但意味着「给 AI 一句话生成可交互的室内场景」离实用近了一大步。
📄 原文摘要(英文)
Diverse and simulation-ready indoor scenes are essential for interactive entertainment and embodied AI, yet their scalable generation remains challenging. Recent agentic text-to-3D scene pipelines that rely on vision-language models (VLMs) can generate scenes of high fidelity but require costly iterative object placement and refinement. Another mainstream paradigm, parametric image-to-3D scene models, produces scenes efficiently from strong priors learned from 2D images but often leads to imprecise and physically invalid scenes. More importantly, both paradigms struggle to output diverse scenes for a single input, making it hard for them to reflect the dynamically changing nature of real scenes. In this paper we propose SceneMosaic, a framework that combines the merits of both paradigms. It obtains the initial candidate from the learned image-based prior, and subsequently evolves the result through VLM agents, ensuring both efficiency and physical validity. Within the evolution process, SceneMosaic exploits the locality of natural scenes and decomposes a scene into independent local units, allowing separate evolution within each unit before composing the global scene via Cartesian product. On SceneEval-100, SceneMosaic matches the strongest agentic baseline in semantic layout quality with a 24x speedup, substantially reduces physical violations, and receives the highest human ratings. Our code is publicly available at https://github.com/rxjfighting/SceneMosaic.