AI终于能像人一样「边看边想」了
现在的视觉AI有个怪毛病：让它分析一张复杂图片它很在行，但让它实时看一段视频、边看边反应，它反而又慢又笨。这篇论文直接拆了传统架构——核心是自研的Mage-ViT编码器，它不再每帧都完整处理，而是只挑画面里「信息量大的区域」编码，像人眼一样只关注动态变化的部分。结果：视觉token消耗减少75%以上，推理速度提升3.5倍，同时视频理解和空间推理能力反而更强。更妙的是，它模仿人脑设计了双系统：一个轻量级「事件门」快速判断要不要处理当前画面，一个深度解码器负责精细理解。这不是你明天能用的工具，但它指明了方向：AI的实时感知能力正在逼近人类。
📄 原文摘要(英文)
Standard vision-language models (VLMs) suffer from Moravec's paradox: they excel at complex offline visual reasoning but struggle with simple streaming perception tasks and process them inefficiently. We present Mage-VL, an efficient codec-native streaming foundation model for real-time multimodal understanding and interaction. At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions using motion vectors and residual energy across sparse anchor (I) and predicted (P) frames. Operating at a 16 x 16 patch level, this reduces visual token consumption by over 75% while preserving spatiotemporal context. Trained from scratch on approximately 560M unlabeled images and 100M unlabeled video frames, Mage-ViT matches or outperforms flagship encoders trained on billions of image-text pairs. We establish AI4AI data pipelines encompassing prompt-code joint optimization for multimodal captioning and AI-driven performance diagnosis to guide training recipes. Furthermore, through a bio-inspired dual-system architecture - a lightweight System 1 event gate and a causal System 2 decoder - Mage-VL enables proactive streaming perception. Extensive evaluations show that Mage-VL-4B matches Qwen3-VL-4B on static tasks while achieving strong gains in video understanding and 2D/3D spatial reasoning, with up to a 3.5x wall-clock inference speedup, and comprehensively surpasses the 15B Phi-4-reasoning-vision baseline. Beyond model artifacts, we deliver seven key empirical findings covering pre-training data efficiency, variable-resolution scaling, codec system acceleration, VideoQA SFT redundancy, motion-spatial synergy, AI4AI data pipelines, and Zero-Vision SFT for multimodal RL.