AI 终于能像真人一样边听边看边回应
现在的 AI 对话要么只能听声音、要么只能看文字，而且得等你把话说完它才开始想。这篇直接把「看画面、听声音、说人话」塞进一个模型里，不用拆成语音识别、语言模型、视频生成好几个模块来回倒腾。它用了一种叫「块因果注意力」的机制，让 AI 可以一边接收你的视频和声音、一边实时生成回应，最低 160 毫秒就能输出一帧，模型端延迟约 200 毫秒，加上网络延迟总共 550 毫秒——已经接近真人对话的节奏。这不是你明天能用的产品，但它让「AI 像真人一样边看你的表情边听你说话边回应」这件事从架构上变得可行。
📄 原文摘要(英文)
We present Wan-Streamer, a native-streaming, end-to-end interactive foundation model designed from the ground up for real-time, low-latency, full-duplex audio-visual interaction. Wan-Streamer seamlessly models language, audio, and video as both input and output within a single Transformer, where the sequence is represented as interleaved visual, audio, and text input tokens together with visual, audio, and text output tokens, coordinated by block-causal attention for incremental streaming. Unlike cascaded interactive systems that rely on separate VAD, ASR, language, TTS, audio-driven animation, or video-generation modules, Wan-Streamer does not rely on external language, speech, avatar, or video-generation modules: perception, reasoning, generation, response timing, turn management, and cross-modal synchronization are learned jointly within one unified model, reducing pipeline latency and error accumulation. To support natural audio-visual responsiveness, we redesign the entire stack around streamability, including causal encoders, causal decoders, block-causal attention, and low-latency multimodal token scheduling, enabling streaming units as short as 160 ms at 25 fps. Wan-Streamer achieves approximately 200 ms model-side response latency and approximately 550 ms total interaction latency when combined with 350 ms bidirectional network latency, supporting sub-second duplex audio-visual communication. These results position Wan-Streamer as a unified, end-to-end, multimodal interactive foundation model for low-latency streaming interaction.