AI终于能告诉你视频里的关键片段在哪
现在的AI能描述视频里发生了什么,但很少能指出关键证据出现在哪一秒。这篇研究让一个模型同时处理不同长度、不同领域、不同提问方式的视频,并输出多个时间区间。它用了一种新方法:先让AI自己生成候选片段,再通过多个模型投票和边界修正来构建可靠标注,最后用新的奖励函数(Wasserstein距离)来训练,避免传统方法对重叠片段惩罚不准确的问题。在7个基准测试上,2B参数的版本就超过了所有同尺寸模型,8B版本甚至超越了397B参数的开源模型。它不是你明天就能用的工具,但意味着未来的视频搜索、内容审核、教学回放等场景,AI能更精准地定位到你需要的那几秒。
📄 原文摘要(英文)
Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a variable-cardinality set of evidence intervals across video lengths, domains, query forms, and viewpoints. Existing training strategies are misaligned with this set-valued task: long-video labels often rely on brittle one-pass annotation, while reinforcement-learning rewards either fail to distinguish non-overlapping predictions or require fragile segment matching. TimeLens2 treats temporal evidence as an interval set throughout supervision and optimization. TimeLens2-93K constructs reliable multi-span supervision through caption-derived proposals, independent localization, cross-agent consensus, semantic verification, and boundary refinement. Our temporal Wasserstein reward computes exact one-dimensional \(W_1\) between uniform distributions over merged interval supports, providing dense, matching-free feedback under unequal cardinalities and equivalent fragmentation; temporal IoU complements it with precise-overlap feedback. Across seven benchmarks, TimeLens2-2B outperforms all size-matched baselines on every benchmark, while the 4B and 8B variants achieve state-of-the-art performance, surpassing open-source models with up to 397B parameters. The 2B, 4B, and 8B variants improve over their Qwen3-VL backbones by 14.2, 13.0, and 18.1 mIoU points, respectively.