AI Pulse
📄 论文解读

AI虚拟人头像的糊与歪,有了修法

用手机视频重建的 3D 数字人头像,一直有个老毛病:侧面、张嘴这些没拍到的角度会糊、会拉长、会抖。这篇的做法不是逐帧修图,而是先用扩散模型把多视角渲染修干净,再拿修好的画面反过来重训那个头像本身——修一次,之后每帧都受益。身份特征单独注入,保证修完还是你那张脸;内部按退化类型分工的专家模块,应付不同程度的糊。在三种重建方法、各三种设置下,感知质量和重建指标都超过原始输出和逐帧修图基线,大角度侧脸和张嘴表情也扛住了。做虚拟主播、远程会议头像、数字人内容的人,这是能直接接进现有流程的一步。

📄 原文摘要(英文)

Recent advances in Gaussian splatting have enabled efficient and photorealistic head avatar reconstruction, yet monocular video-based or image-based methods still suffer from under-constrained geometry and texture, leading to artifacts such as blurring, stretching, and temporal instability. We present EDGAR (Efficient Diffusion-Driven Gaussian Avatar Refinement), a conditional diffusion framework designed to enhance the quality of Gaussian avatars by correcting artifacts while preserving identity fidelity. To train EDGAR, we construct EDGAR-SET, a hybrid triplet dataset composed of real-captured and synthetic-rendered samples, which simulates realistic Gaussian avatar failure cases by pairing degraded renderings with ground-truth and reference images. EDGAR refines degraded frames through a fusion block that injects identity features extracted by the self-supervised Sapiens model into a one-step diffusion backbone, enabling effective artifact removal without sacrificing efficiency. Our EDGAR model incorporates a Mixture-of-Experts (MoE) architecture to address diverse degradation types and severity levels. The refined multi-view renderings are subsequently used as pseudo-targets to optimize a single underlying Gaussian avatar, so that the final avatar can be rendered without applying EDGAR to every output frame. Experiments on three different Gaussian avatar reconstruction methods, each evaluated under three different settings, show improvements in both perceptual and reconstruction measures over the raw outputs and an SDEdit-style refinement baseline. In addition, cross-view evaluation demonstrates improved calibrated consistency, while tests under large yaw and open-mouth expressions verify robustness under extreme settings. These results collectively indicate a useful operating regime for recoverable appearance artifacts. The code is available at https://kwanyun.github.io/edgar/

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新