一张照片,生成能换光的 3D 模型
现在的 AI 从一张图生成 3D 模型,基本是「死」的:光一换就假,材质像塑料。这篇把「模型长什么样」和「它是什么材质」拆开存,生成时同时给出颜色、粗糙度、表面法线,于是你能在渲染软件里像调真东西一样换灯光、改反光,连文字和 Logo 都能保住。它不是你明天就能用的工具,但这是 3D 资产从「能看」到「能用」的关键一步。
📄 原文摘要(英文)
High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.