AI Pulse
📄 论文解读

AI 助手连「记得上周的事」都做不到

现在的 AI 助手评测,都是「你问一句、它答一句」的短对话。但真实生活不是这样:你让它帮你安排一个月的搬家,它得自己盯着时间、发现没人告诉它的变化、在合适的时候开口。这篇论文造了一个模拟世界,里面有 22 个假服务,给 7 个最强模型布置了 200 个跨几周的任务,结果全部低分。它不是你明天能用上的东西,但它划了一条线:现在的 AI 离「生活助理」还差得远,差在主动性,不在聪明。

📄 原文摘要(英文)

Large language model (LLM) agents are increasingly deployed as personal assistants. Existing evaluations, however, mostly use short, self-contained requests in static environments. Everyday life assistance is different. A task runs for weeks rather than minutes. The world keeps changing while the agent is not being prompted. Many constraints are never stated outright. An agent that merely answers the request in front of it will fail at such a task. What is needed instead is an agent that stays proactive and consistent. It decides on its own when to act, when to ask, and when to stay silent. It notices changes that nobody announced. It keeps one plan coherent from the first day to the last. No current benchmark measures this. We introduce VibeLifeBench, a benchmark of 200 long-horizon tasks across ten everyday-life domains. Each task is a scripted multi-week timeline in a simulated world of 22 mock services. The world advances on its own clock, and many of its changes are silent, so only an agent that re-inspects the world discovers them. Every task is graded by fine-grained, weighted checks that read only what the agent actually left behind, covering the end state, the timeliness of its actions, and whether it upheld the implicit constraints. We evaluate seven frontier models. All of them score low, which shows how far current agents are from assisting with real life. We will open-source all tasks, environments, and the evaluation framework.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新