AI Pulse
📄 论文解读

视频编辑不再只改旧片,能边拍边改

现在的 AI 视频编辑有个默认前提:你给它一段完整的视频,它从头到尾改一遍。但直播、游戏画面这种没完没了的流,根本不存在「完整」——你改的时候,新画面还在不断进来。这篇把问题反过来:给定前面一段和一句编辑指令,模型要生成「接下来」的片段,既延续原画面、又带上你要的改动,而且指令可以一条接一条来,改完一版还能再改下一版。做法是给现成的流式视频生成器加一个轻量适配器,里面三个注意力模块各管一件事:看历史帧、只让时间信息从前往后流、把编辑指令注入。关键设计是,适配器只在收到指令的那一小段激活,之后恢复原模型继续无限生成,所以编辑不会越积越乱。实验里它在连续多轮编辑下保持稳定。这不是你明天能拿来剪片的工具,但它把「视频编辑」从改旧片变成了「边生成边改」,直播、游戏、监控这类实时场景才是它的主场。

📄 原文摘要(英文)

With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption. They align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. In this paper, we study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新