AI Pulse
📄 论文解读

AI终于能像人一样边看边想,不再卡顿

现在的视觉AI有个怪毛病:做复杂的图像推理(比如看图写作文)很厉害,但处理简单的实时画面(比如看视频里物体移动)反而又慢又差。这就像一个人能解微积分,却接不住飞来的球。研究者从视频压缩技术里偷师,让AI只盯着画面里变化最大的区域(比如移动的物体边缘),而不是每一帧都从头到尾扫一遍。结果视觉信息消耗减少了75%以上,推理速度提升了3.5倍,在视频理解和空间推理上还更强了。这不是你明天就能用的工具,但它指出了方向:未来的AI眼镜、自动驾驶、实时翻译,可能不再需要笨重的云端计算,而是像人眼一样,只抓重点、快速反应。

📄 原文摘要(英文)

Standard vision-language models (VLMs) suffer from Moravec's paradox: they excel at complex offline visual reasoning but struggle with simple streaming perception tasks and process them inefficiently. We present Mage-VL, an efficient codec-native streaming foundation model for real-time multimodal understanding and interaction. At its core, our custom tokenizer, Mage-ViT, replaces uniform frame sampling by selectively encoding dynamic, entropy-rich regions using motion vectors and residual energy across sparse anchor (I) and predicted (P) frames. Operating at a 16 x 16 patch level, this reduces visual token consumption by over 75% while preserving spatiotemporal context. Trained from scratch on approximately 560M unlabeled images and 100M unlabeled video frames, Mage-ViT matches or outperforms flagship encoders trained on billions of image-text pairs. We establish AI4AI data pipelines encompassing prompt-code joint optimization for multimodal captioning and AI-driven performance diagnosis to guide training recipes. Furthermore, through a bio-inspired dual-system architecture - a lightweight System 1 event gate and a causal System 2 decoder - Mage-VL enables proactive streaming perception. Extensive evaluations show that Mage-VL-4B matches Qwen3-VL-4B on static tasks while achieving strong gains in video understanding and 2D/3D spatial reasoning, with up to a 3.5x wall-clock inference speedup, and comprehensively surpasses the 15B Phi-4-reasoning-vision baseline. Beyond model artifacts, we deliver seven key empirical findings covering pre-training data efficiency, variable-resolution scaling, codec system acceleration, VideoQA SFT redundancy, motion-spatial synergy, AI4AI data pipelines, and Zero-Vision SFT for multimodal RL.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新