AI在陌生世界里探索，最强模型也会越探越差
科学发现不是解已知题，而是去未知里找新问题。但怎么判断AI是真探索出了新规律，还是背了训练数据里的旧答案？这篇论文造了两个「外星世界」:规则是能跑的代码，每个答案都能精确验证，而且规则故意和常识冲突——背答案没用，只能真去试。10个AI系统进去探索，最强的那批能学会并应用陌生规则，但表现很不稳定：同一条探索路径，结果忽好忽坏；继续探索有时不但没进步，反而把之前学到的东西搞丢了。它不是你明天能用上的东西，但它把「AI能不能做真正的科学发现」从口号变成了可检验的考题。
📄 原文摘要(英文)
Scientific discovery begins where known problems end. There, AI systems must engage in exploration: framing hypotheses, designing experiments, and iterating on the results. However, evaluating this ability is difficult: (1) how to verify whether a genuinely new hypothesis holds, and (2) how to determine whether a system has discovered it through exploration or merely recalled related knowledge from pre-training data. To this end, we introduce ExplorationBench, which turns the wicked problem of evaluating scientific exploration into a concrete and tractable framework built on verifiable Alien Worlds: their rules are executable, so every answer can be checked exactly, and they conflict with familiar knowledge, so recall alone cannot solve the tasks. The benchmark contains two sandboxes, AlienCode (31 discovery targets, 70 tasks) and AlienLogic (24 discovery targets, 70 tasks). Each sandbox provides a flawed manual, task-specific environmental feedback, and a dedicated tool-call schema. Systems use these resources to explore the sandbox, then solve held-out tasks. We evaluate 10 AI systems and find that the strongest systems can acquire and apply unfamiliar rules, while performance varies substantially across trajectories and continued exploration can stall or reverse earlier gains. ExplorationBench represents a step towards AI systems that can acquire and apply genuinely new knowledge through exploration in unknown environments.