AI Pulse
📄 论文解读

一张照片,生成能换光的 3D 模型

现在的 AI 从一张图生成 3D 模型,基本是「死」的:光一换就假,材质像塑料。这篇把「模型长什么样」和「它是什么材质」拆开存,生成时同时给出颜色、粗糙度、表面法线,于是你能在渲染软件里像调真东西一样换灯光、改反光,连文字和 Logo 都能保住。它不是你明天就能用的工具,但这是 3D 资产从「能看」到「能用」的关键一步。

📄 原文摘要(英文)

High-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using dedicated Gaussian primitives for each modality. A variational autoencoder compresses this representation into a unified material-aware latent space. A rectified-flow transformer generates this latent from a single image, conditioned on multi-layer features from a pretrained image encoder that preserve both semantic context and fine spatial detail. The latent then decodes into relightable PBR Gaussians and an optional textured mesh with a tangent-space normal map. On Toys4K, Luce achieves state-of-the-art single-image-to-3D generation, improving FID by 28% over the strongest baseline. We further introduce a benchmark of AI-generated images, on which Luce improves the CLIP image-alignment score over the best baseline (0.8519 vs. 0.8299). Luce generates relightable, geometrically accurate, and materially faithful assets that preserve fine details such as text, logos, and inscriptions.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新