AI 的品味，成了新的能力瓶颈
大模型做长任务时，中途的每个选择——先测哪个假设、在哪个代码上继续搭——决定了整件事的成败。研究者把这种中途决策能力叫做「品味」，并造了个评测集：把 AI 自己跑任务时的轨迹切成岔路口，让模型在不知道后续结果的情况下选方向。最强模型只答对 59.7%，而且岔路口的线索出现得越晚，所有模型越抓瞎；给更多推理时间也没用。好消息是品味可以教：让看过结局的老师把判断蒸馏给学生，学生在没见过的任务上决策更好，端到端成功率也涨了。它不是你明天能用上的，但它在提醒：AI 的下一个瓶颈可能不是算力，是判断力。
📄 原文摘要(英文)
LLM agents increasingly work on long-horizon tasks, and the decisions they make along the way, such as which hypothesis to test or which implementation to build on, determine the outcome of the whole run. Making these decisions well is becoming a key capability for both engineering and research agents. We refer to the ability to make good long-horizon decisions as the taste of an agent. While existing benchmarks measure the end-to-end success of agents on long-horizon tasks, none of them measures the taste of an agent. To address this problem, we build Taste-Bench, a benchmark of taste questions constructed automatically from trajectories that agents produced in engineering and research tasks. Each question presents a decision fork, a point in a trajectory where multiple directions are available and one of them leads to a better outcome, and the evaluated model chooses among these directions without seeing what happens after the fork. We mine these forks automatically from parallel attempts at the same task and from detours inside a single trajectory, without needing human annotation. We evaluate frontier models on Taste-Bench and find that the best model answers only 59.7% of the questions correctly. We further find that forks whose deciding evidence appears later in the trajectory are much harder for every model, and that a larger reasoning budget does not improve the accuracy. Finally, we show that taste can be trained. We distill the judgment of a teacher that has seen the outcome into a student model, and the student makes better decisions on unseen tasks and improves end-to-end success on held-out SWE-bench Pro tasks.