一张照片变3D场景,还能随意转视角
现在的AI能从一张照片生成新视角,但要么转角度大了就变形,要么只能小范围动。MetaView把两种思路捏在一起:它用神经网络先估算出照片里物体的深度和大致形状(这叫隐式几何先验),再结合一个关键的“真实尺度”信息——比如照片里一个杯子实际多高。这样,当你把视角转到侧面甚至背面时,AI知道物体该长什么样、该有多大,不会扭曲或飘走。实验里,在超过90度的大角度旋转下,它生成的画面比现有方法清晰、真实得多。它不是你明天就能用的App,但做3D建模、虚拟试穿、游戏场景的人,可以关注这个方向——未来可能一张自拍就能生成你的3D模型。
📄 原文摘要(英文)
Current visual generation models are capable of producing high-quality content, yet they lack a coherent perception of the spatial structure. Existing generative novel view synthesis methods typically introduce explicit geometry priors, which enforce spatial consistency but inherently restrict generalization in large view changes. In contrast, recent interactive generative methods favor implicit scene modeling, offering greater flexibility at the cost of precise camera control and geometry consistency. In this paper, we propose MetaView, a diffusion-based monocular novel view synthesis framework that enables rendering under large view changes from a single image. Our key insight is to combine implicit geometry modeling with minimal yet essential explicit 3D cues: we incorporate implicit geometry priors from a feed-forward geometry perception network to regularize structure without imposing restrictive reconstruction pipelines, while leveraging metric depth to anchor the generation to a metric scale. This design allows MetaView to achieve both geometry consistency and precise controllability. Extensive experiments demonstrate that, under challenging monocular large viewpoint changes, MetaView significantly outperforms existing methods and exhibits superior generalization. Our code is publicly available at https://github.com/KlingAIResearch/MetaView.