视频生成进入实时时代:边看边改,人物还能听你指挥跳舞
视频生成一直有个尴尬:你等它生成完,发现不满意,又得从头再来。Vidu S2 把这件事变成了「直播」——它能在你眼前实时生成 720p 视频,而且你随时可以换参考图、改指令,画面立刻跟着变。更狠的是,它支持实时编辑视频流:换衣服、换背景、换角色、改画风,都是边播边改,不用等渲染。人物还能听懂「跳个舞」这种指令,动作跟着来。这不是论文里的概念演示,官方直接放出了可玩的在线 demo。对做直播、短视频、虚拟偶像的人来说,这可能是第一个真正能「边聊边生成」的工具。
📄 原文摘要(英文)
We present Vidu S2, which comprises Vidu S2-Avatar, a real-time interactive digital-character model, and Vidu S2-Editing, a real-time video editing model. Moreover, we explore the feasibility of real-time spatial video generation for both Vidu S2-Avatar and Vidu S2-Editing. Compared with Vidu S1, Vidu S2-Avatar supports real-time 720p video generation, generation with dynamic references that can be updated at any moment, and stronger instruction following, such as dancing. Vidu S2-Editing supports editing a video stream in real time, including style rendering, clothing replacement, character replacement, and background replacement. Experiments show that Vidu S2 outperforms all baselines. A playable online demo is available at https://vidu.com/vidu-stream.