AI看视频的短板：记不住一小时前的事
现在的AI看视频评测，大多只给几十秒的片段和选择题，导致一个只处理最后四帧的简单模型就能和复杂模型打平——因为答案选项本身就泄露了线索。这篇论文做了个更狠的测试：让AI看平均88分钟的长视频，然后回答开放式问题，考察它能不能实时感知、回忆很久以前的事、主动提问、调用工具。结果发现，只记最近几帧的模型记不住远事，把过去转成文字的模型丢视觉细节，反复压缩记忆的模型保不住精细信息。作者用双层架构解决：前端负责即时反应，后端异步建长期记忆、做历史回忆和外部搜索，四个能力都超过现有基线。它不是你明天能用上的东西，但它点破了一个真问题：AI的“记忆”和“反应”是打架的，而长视频才是真实世界的常态。
📄 原文摘要(英文)
Deploying autonomous multimodal agents in continuous, real-world environments requires them to ingest unbounded audio-visual streams and maintain hour-scale memory. However, current evaluations predominantly rely on brief clips and multiple-choice formats. This design allows minimal baselines that process only the last four frames to match or surpass complex streaming models, while answer options also expose language shortcuts. We introduce StreamArena, a benchmark for hour-scale, interactive streaming video understanding. StreamArena contains 243 full-length videos averaging 88.8 minutes and 3,646 rigorously annotated, open-ended question-answer pairs that evaluate real-time perception, historical retrospection, proactive interaction, and multimodal tool utilization. Evaluation across diverse systems exposes a tension between continuous interaction and long-horizon multimodal comprehension. Methods that retain only recent frames cannot recover distant events, methods that convert past observations into text lose visual evidence, and methods that repeatedly compress visual memory struggle to preserve fine-grained details over time. We address this tension with StreamMind, a two-tier architecture that assigns latency-critical interaction and proactive monitoring to independently scheduled frontend workers, while backend workers asynchronously construct persistent multimodal memory and perform historical recall and external search. StreamMind outperforms existing streaming baselines across all four capabilities and reduces query-to-answer latency by reusing persistent state.