视频编辑不再限时长,能一直改下去
现在的 AI 视频编辑有个隐藏前提:你给一段固定长度的视频,它从头改到尾。但如果你在直播游戏、或者拍一个持续的长镜头,编辑指令是不断到来的——比如「把画面风格改成赛博朋克」,过一会又说「加个镜头平移」——现有模型就卡住了,因为它只会对着整段素材重算,没法接着往下编。这篇论文把这个问题叫「无限视频编辑」,并给出了一个轻量方案:在生成下一段时,只用一个适配器模块去「记住」前面已经改好的画面和编辑指令,然后让模型继续生成,而不是回头重写。这个适配器只在你下达新指令的那一段被激活,之后又切回原始模型,所以不会越改越崩。实验证明,它能稳定处理连续几十次编辑指令而不退化。它不是你明天就能用的产品,但它指向一个方向:视频编辑工具会从「改一段」变成「改一场直播」,从剪辑软件变成实时导演台。
📄 原文摘要(英文)
With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption. They align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. In this paper, we study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.