一段手机视频，就能生成可旋转的3D人
你手机随便拍一段人的视频，现在能直接变成可以360度旋转、从任何角度看的3D模型了。过去这类技术要么需要多台相机同步拍摄，要么生成的侧面和背面会扭曲变形。这篇论文的核心突破是解决了AI在生成多视角画面时的「注意力瓶颈」:当要生成的视角太多时，模型会顾此失彼。他们用两个技巧——把参考画面压缩成固定大小的混合分辨率信息包，以及在去噪过程中轮换不同视角组的信息交换——让模型能稳定生成几十个视角都一致的画面，再把这些画面拼成完整的4D高斯模型。在标准测试集上，新视角视频质量和3D重建效果都明显优于此前方法。
📄 原文摘要(英文)
We present 4DAnyone, a framework for reconstructing 4D humans from an uncalibrated monocular video by generating reconstruction-grade multiview-consistent videos and lifting them into 4D Gaussian Splatting (4DGS). Existing camera-controlled video diffusion models synthesize plausible novel-view videos but fail to maintain consistency when scaled to the tens of target views required for 4DGS reconstruction. We identify this failure as a bounded-attention-context problem: when target views exceed the capacity of a single DiT forward pass, they must be split into groups, exposing two coupled bottlenecks. On the reference-context side, conditioning on all previously generated views grows as O(N), weakening cross-view appearance guidance. On the target-context side, disjoint groups cannot directly exchange information, causing global structural drift. 4DAnyone addresses both bottlenecks with two complementary designs: Reference Context Packing (RCP) compresses growing reference views into a fixed-length mixed-resolution context with O(1) reference-context complexity, while Target Context Routing (TCR) rotates target-view groupings during denoising to share context across groups at high-noise steps and stabilize details at low-noise steps. We further build the MVGameHuman dataset using our in-house game engine and combine it with light-stage and in-the-wild video datasets for training. Experiments on DNA-Rendering and DyMVHumans show that 4DAnyone outperforms prior methods in both novel-view video quality and downstream 4DGS reconstruction, with robust in-the-wild generalization. See our project page for video results and source code: https://4danyone.github.io.