让视频里的角色真的转起来,不再一转身就崩
现在的 AI 生成角色视频,镜头只能小幅度动:一旦镜头大角度绕人物转,角色背面、侧面这些开头没露过的部分就会崩——脸变形、衣服糊、人像换了个人。这篇把「镜头怎么转」和「角色长什么样」拆开处理:先用第一帧估算出 3D 点云,按目标镜头轨迹渲染出几何骨架,再在生成时用镜头相对位置做引导,从多视角参考里取对应角度的特征补全。结果是镜头可以大幅旋转、平移、拉近拉远,角色从头到尾保持同一张脸、同一套衣服。它不是你明天就能用的工具,但这是「AI 视频里角色终于能转身」的关键一步。
📄 原文摘要(英文)
Precise camera control is essential for cinematic character video generation, yet existing methods struggle to follow large-angle trajectories with combined rotation, translation, and radius variation from arbitrary initial azimuths. Such viewpoint changes also reveal character regions unseen in the initial frame, causing geometric distortions, appearance degradation, and identity drift. To systematically study these coupled challenges, we introduce CLACT, a large-scale Character-centric dataset and benchmark covering diverse Large-Angle Camera Trajectories, with accurate camera geometry and multi-view character observations for training and evaluation. We further propose WorldCast, a framework for world-consistent character video generation under diverse large-angle camera motions. Specifically, we back-project the initial frame into a 3D point cloud using estimated depth, align the point-cloud scale with the target camera trajectory, and render it along the trajectory to obtain explicit geometric guidance and coverage masks. To complement the incomplete character information, we introduce Geometry-Aligned Memory Attention (GAMA). Using relative camera poses estimated jointly from the initial frame and multi-view character anchors, we express anchor and target-view geometry in a common coordinate frame. GAMA uses this geometric alignment to guide the retrieval and injection of view-relevant character features during generation. Evaluations across diverse large-angle camera trajectories demonstrate accurate trajectory following while preserving character geometry, appearance, and identity.