AI Pulse
📄 论文解读

超分重建选错“空间”,换一个就快了 27 倍

把模糊小图变清晰,过去都在“像素空间”或“重建专用空间”里做,就像在乱糟糟的仓库里找东西。这篇发现:换到一种“视觉基础模型”的内部空间,模糊图的坐标离清晰图集合其实很近,而且这个空间自带从细节到语义的分层信息,模型能顺着它找回原图。他们用这个思路训练了一个 4 亿参数的模型,在 4 个真实超分基准上,清晰度和观感平衡都超过现有最强方法,而且一张 512×512 的图只要 37 毫秒——比同类扩散模型快一个数量级。这不是你明天能用的工具,但它指了个方向:很多图像任务卡住,可能不是模型不够大,而是选错了干活的空间。

📄 原文摘要(英文)

In an image latent space, the embeddings of high-resolution, natural, and sharp images form a manifold. Degradation of high-resolution images pushes their embeddings off this manifold. Real-world super-resolution (SR) then becomes the task of mapping the degraded embedding back onto this manifold --- not anywhere on the manifold, but to the point that preserves what the input still carries, both its semantics and pixel details. Every published method implements this mapping in a reconstruction-oriented latent space or pixel space. We claim these spaces are the wrong substrates for SR. Low-resolution and degraded images are embedded far from the manifold, making the mapping difficult and expensive. The lack of semantic information in these substrates also makes it difficult to navigate to the faithful point on the manifold, causing severe hallucination when degradation is heavy. Thus, restoring in a suitable latent space is crucial to the SR task. We show that the latent space of 23 fused layers of a frozen DINOv3-L is one such space that makes the SR task easier. Degraded images are embedded near the manifold. Moreover, this substrate contains a hierarchy of information, from pixel record to degradation robust semantics, guiding the model to find the faithful point on the manifold. On this substrate, a 415M decoder is trained under reconstruction and adversarial objectives to map the degraded embeddings back and decode to pixel space in one pass. The resulting model, RAESR, attains the best fidelity--perception trade-off among state-of-the-art adversarial and diffusion-based restorers on RealSR, DRealSR, LSDIR and DIV2K-Val, at 37 ms per 512 by 512 image on a single H20 GPU. Swapping the substrate for a VAE latent under an identical recipe loses on every metric.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新