AI Pulse
📄 论文解读

AI视频终于能记住主角长什么样了

现在的AI视频生成,主角换个镜头就换张脸,这是它没法拍长故事的根本原因。JoyAI-Echo-1.5把「记忆」装进了视频模型:它不只看着当前这一帧,还会回头翻前面几段画面,把主角的长相、声音特征攒起来,再决定下一段怎么拍。同时它把「镜头怎么动」从文字里拆出来,变成精确的3D轨迹,你按手柄、拖鼠标、打字都能控制视角。结果是同一角色能跨几十个镜头保持长相和声音一致,画面质量也压过现有方案。它还不是你明天能用的产品,但「视频模型有记忆」这件事,是长视频和可交互世界真正能成立的前提。

📄 原文摘要(英文)

Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts. We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants. The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio, enabling persistent character appearance and voice identity across flexible combinations of text, image, and memory conditioning. The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway, enabling controller-agnostic interaction across flexible viewpoints. To support efficient long-horizon generation, we transform a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. Experiments demonstrate strong performance in both settings. JoyAI-Echo-1.5 achieves improvements over existing long-video baselines in cross-shot consistency, visual quality, text alignment, and speech fidelity. Its world-model variant ranks first on WBench, with an average score of 81.7, and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench. Together, these results indicate that memory, geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds. Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新