AI视频助手评测：能看会听，但不会干活
你让AI当你的实时视频助手——比如一边做菜一边问它下一步该放什么——它看起来能看能听，但一上手就露馅。这篇评测搞了个新基准OmniAssistBench，不是让AI看一段视频然后答题，而是模拟真实互动：AI得边看你的操作边给指令，你的下一步又取决于它的回答。为了公平比较，他们从网上视频反推出用户目标和多轮对话，花了1000个专家工时。结果：最强闭源模型Gemini-3-Pro只拿了66.4分（满分100），开源模型Qwen3-Omni-Instruct是51.2。问题很具体：看不懂手势、记不住前面说了啥、该等的时候抢着答。它不是你明天能用上的助手，但告诉你一个事实：现在的多模态大模型，离“靠谱助手”还差一大截。
📄 原文摘要(英文)
Recent omni-modal large language models (Omni-LLMs) show great potential as real-time video assistants, which continuously perceive environments and guide users to achieve specific goals. Unlike traditional passive video understanding, interactive assistants should actively combine visual states, user goals, and prior knowledge to provide effective help. Evaluating this is rather challenging, as the model's unpredictable response dynamically changes the user's subsequent actions, which static offline datasets cannot accommodate. To address this bottleneck, we introduce OmniAssistBench. To solve the issue of diverging interaction paths where the same user goal can be achieved through various methods, we provide models with predefined priors derived from the source video, requiring them to guide users along the exact same routes. Since real interaction videos are rare, we construct the dataset by reverse-engineering existing Internet videos. We deduce logical user goals and segment the videos into multi-turn clips to simulate continuous interactions. This rigorous pipeline required over 1000 expert person-hours to build the dataset. Results show that the proprietary Gemini-3-Pro reaches 66.4 out of the max point of 100, while the open-source Qwen3-Omni-Instruct achieves 51.2. Although current models generally understand user inputs, they frequently provide incorrect or incomplete answers. Specifically, they struggle with visual prompts (e.g., hand gestures), fail to maintain historical context during multi-turn interactions, and fail to delay response until the target event. Results indicate substantial room for improvement before models can become reliable assistants.