AI Pulse
📄 论文解读

AI实时对话画面从模糊变清晰,延迟不变

你正在和AI视频聊天,它能看到你、听到你,但画面一直是模糊的——因为高清视频实时生成太慢。现在有个叫Wan-Streamer的模型升级到了v0.2,把输出分辨率从192x336提升到640x368,清晰度翻了几倍,但延迟依然保持在200毫秒左右(模型处理时间)。怎么做到的?它把AI拆成两个角色:一个“思考者”快速理解你的动作和语言,一个“表演者”用多张显卡并行生成高清画面。思考者只负责发指令,表演者内部把长视频拆成小块分给不同显卡处理,最后拼起来。这样,高清视频的生成时间被分摊,用户感觉不到卡顿。加上网络传输,总延迟约550毫秒,依然适合实时对话。

📄 原文摘要(英文)

We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

arXiv 原文

📬 订阅 AI Pulse

每天三次更新,不错过重要信号

▲ 回到顶部