3D场景生成与重建，终于不用先压缩再解压了
现在的3D生成和重建通常分两套做法：重建靠像素回归，生成靠潜空间扩散。但潜空间方法有个硬伤——信息先被压缩再解压，细节和几何结构都会丢，还得额外训练一个编码器。PixWorld直接把扩散过程放在渲染后的图像上，用图像级别的损失来监督，同时引入一个3D基础模型的几何感知损失，让模型在生成时也能理解三维结构。结果：生成质量超过所有潜空间方法，重建精度追平最先进的重建专用模型。它不是你明天就能用的工具，但说明了一个趋势：3D AI正在从“先压缩再想象”转向“直接在像素里想象”，这对未来做游戏、影视、数字孪生的人是个好消息。
📄 原文摘要(英文)
3D reconstruction and generation are commonly tackled by separate paradigms: pixel-based regression for reconstruction, and latent diffusion for generation. Recent works attempt to unify them in latent space, but with notable drawbacks: the diffusion objective is defined on latent features rather than the underlying 3D representation, and both branches suffer from information loss introduced by latent encoding, while requiring a pretrained Variational Autoencoder (VAE) or Representation Autoencoder (RAE). In this paper, we reformulate these two tasks under a unified pixel-space diffusion paradigm and introduce PixWorld, a single model that jointly addresses 3D reconstruction and generation. By supervising diffusion directly on rendered images, PixWorld removes the above limitations and aligns optimization with 3D scene fidelity. Beyond photometric and perceptual supervision that operates at the 2D image level and lacks 3D geometric awareness, we further introduce a geometry perception loss that aligns rendered views with their ground truth in the geometry-aware feature space of a pretrained 3D foundation model, providing 3D structural supervision. PixWorld consistently outperforms prior latent-space generation methods and matches state-of-the-art reconstruction methods, demonstrating the superiority of a unified pixel-space approach.