AI Pulse
📄 论文解读

把视频模型当画布,一张图能改出无数张

我们习惯把 AI 生图当成「从零画一张」,但 FrameForge 换了个思路:把视频模型里「时间流动」的本事借来,让一张图能按你的要求变成另一张,而且变的方式更聪明。它不把「原图」和「目标图」固定放在时间线的两头,而是让模型自己判断这两张图之间隔了多远、中间经历了什么变化——就像看一段视频,知道中间帧是怎么过渡的。它用两套机制:一套负责「距离感」,另一套负责「推理路径」,让信息不是直接跳过去,而是经过一个中间状态再到达目标。在三个公开测试集上,它在主体驱动生成、边缘图转图和图像编辑上都超过了现有方法。这不是你明天就能上手用的工具,但它指向一个趋势:未来的图像编辑会越来越像「导演剪辑」,而不是「滤镜叠加」。

📄 原文摘要(英文)

Unified conditional image generation requires a single model to preserve different aspects of a visual input while producing a target image that follows textual guidance. Pretrained video models provide a promising foundation by encoding visual transformations across time, yet existing approaches typically assign the condition and target to fixed temporal positions. We observe that condition-target relationships vary substantially across generation tasks, while video models naturally encode visual transformations at different temporal scales. This motivates us to model conditional generation through input-adaptive virtual temporal relations. We propose FrameForge, which captures two complementary aspects of such relations. Mixture of Temporal Experts (MoTE) models how far the target lies from the condition by dynamically selecting temporal RoPE experts with different virtual intervals. Chain of Temporal Reasoning (CoTR) models how the condition is transformed into the target through a latent intermediate state, with a reasoning-bottleneck attention mask encouraging information propagation through this pathway. Experiments on DreamBench, COCO2017, and GEdit-Bench show that FrameForge achieves superior performance in subject-driven generation, canny-to-image generation, and image editing among the compared methods.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新