AI考了600级职业考试，及格率不到一半
让AI连续完成同一职业的5级任务，每级必须通过才能解锁下一级——这是新基准CLE的设计，120个任务、600个等级、覆盖89种职业角色，由50多位专家出题。结果：最强系统只通过47.7%的关卡，能完整走完5级的任务仅23.3%。更关键的是，给AI一次失败后的诊断反馈和修改机会，完成率中位数只提升5.3个百分点——说明当前模型几乎不会从错误中积累经验，更像每次从零开始。这不是你明天能用上的东西，但它把「AI能不能越用越聪明」从口号变成了可量化的考试，也暴露了前沿模型在长期学习上的真实短板。
📄 原文摘要(英文)
Continual learning is the ability of LLM agents to improve over time by acquiring, refining, and applying knowledge from experience while retaining useful prior knowledge. It is essential for agents learning over long time horizons, yet current evaluations largely assess individual task performance rather than improvement through accumulated experience. We introduce Continual Learning Exam (CLE), a large-scale benchmark developed by 50+ domain experts to measure experience-driven improvement in realistic professional workflows. CLE comprises 120 human-authored tasks, each organised as a five-level ladder, yielding 600 levels spanning 89 professional roles across 12 occupational domains. Agents must pass each level to unlock the next, with later levels building on experience accumulated during earlier assignments. An evaluation agent assesses each submission against human-authored rubrics. We compare a single-submission setting with one that provides diagnostic feedback after failure and allows one revision. Across 17 harness-model configurations using frontier LLMs, feedback and revision yield a median completion gain of 5.3 percentage points. Yet the best-performing system clears only 47.7% of levels and completes all five levels in just 23.3% of tasks, highlighting the difficulty of sustained learning from experience for existing frontier LLMs. As such, the dataset can also be used to post-train LLMs to strengthen their continual learning capabilities.