开源视频AI新突破：4B参数吊打大模型
开源视频AI模型有个尴尬：要么只擅长特定场景，要么跑起来烧显卡，而且代码、数据藏着掖着。这篇直接摊牌——模型、训练代码、数据集全开源，还只用了4B参数（比主流小几倍），却在通用、长视频、流式视频三大类任务上超过更大模型。怎么做到的？两个关键：一是把3D视觉Transformer“充气”成I3D-ViT，高效处理时空信息；二是自适应帧率技术，看视频时动态跳帧，省算力。同时，他们自建了三个高质量数据集，覆盖不同场景，让模型见多识广。这不是你明天能用的工具，但如果你关注AI视频理解的前沿，这是开源社区的一次重要突破——小模型、全开放、能打。
📄 原文摘要(英文)
Recent advances in video understanding have spanned motion, long video, and streaming interaction, driving this field toward real-world applications. Despite this progress, current open-source models remain limited in several ways. They often struggle to generalize across diverse video types, making them effective only in specific domains. High computational demands further restrict their efficiency and scalability. Moreover, most models are only partially open, with key components such as training code, strategy, or datasets unavailable, which hinders reproducibility and slows community-driven development. To address these issues, we introduce VideoChat3, a fully open, efficient, and generalist video-centric MLLM. VideoChat3 advances video understanding through two complementary designs. For efficiency, we introduce Inflated 3D Vision Transformer (I3D-ViT) and Adaptive Frame Resolution for Streaming Video Perception, which enables efficient spatiotemporal representation and reduces the cost of processing video inputs during training and inference. For effectiveness, we develop a scalable video data synthesis pipeline that curates three diverse, high-quality training datasets: VideoChat3-Academic2M, VideoChat3-LV116K, and VideoChat3-OL617K, covering general, long-form, and streaming video scenarios, improving the model's generalization across domains. By integrating these designs, VideoChat3 achieves a rare balance of broad generalization and computational efficiency. Experiments across general, long-form, and streaming benchmarks demonstrate that VideoChat3 surpasses prior open-source models with equal or larger parameter counts with only 4B parameters and higher efficiency.