AI Pulse
📄 论文解读

AI看视频卡住时,该学会开口问

现在的AI看视频,遇到不确定的信息只会干等或者硬答——等不到就编。这篇提出第三种动作:主动问。研究者做了第一个专门测这个能力的基准,438个视频、2714个手工标注的时间窗口,让模型自己判断该等、该答、还是该问。结果最强模型也只有52%的准确率,说明AI在「知道自己不知道什么」这件事上还差得远。它不是你明天能用上的东西,但它指向一个更本质的方向:AI不该假装全知,而该学会承认缺口。

📄 原文摘要(英文)

Recent advances in multimodal large language models have increased interest in interaction paradigms for video understanding. The evolution from offline to streaming and proactive interaction has gradually shifted models from passive responders toward more proactive agents that can decide when to answer. However, existing proactive paradigms typically assume that the information required to answer a query is complete, leaving models to only wait or answer. In the real world, however, information is often incomplete due to uncertainty from either the video or the user. In proactive paradigms, such uncertainty can cause the model to wait forever or generate highly hallucinatory responses. We therefore argue that proactive interaction should be redefined with a third action: ask, allowing models to actively ask for missing information. To study when and what models should ask before answering, we introduce AskStream, the first multi-turn benchmark spanning video and user uncertainty for streaming video understanding, with four task categories and eight subtasks. It contains 438 videos with 2,714 manually annotated frame-level time windows across three types: wait, ask, and answer. Across both multiple-choice and open-ended settings, even the strongest model attains only 52.18% on our proposed Ask-F1 metric, highlighting substantial limitations in proactive information seeking under uncertainty.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新