给AI装个“小脑”：大模型只做难题，小事交给4B小模型
现在让AI干活，每一步都要问一次大模型，又慢又贵。这篇论文发现，AI在跑任务时，大部分“下一步该调哪个工具”其实是选择题，根本不用惊动大模型。他们训练了一个只有4B参数的小模型来干这些杂活，但小模型会判断自己有没有把握：没把握才把问题交回给大模型。结果在没见过的任务上成功率91.5%，在见过的任务上成本降了八成、速度快了2.5倍，成功率几乎不掉。这不是你明天能用的产品，但它指向一个趋势：未来的AI不是更大，而是分工——大模型当大脑，小模型当小脑。
📄 原文摘要(英文)
Large language model (LLM) agents now fix software, operate computers and serve customers, but their harnesses, the programs that run them, call a large LLM at every step, which makes long tasks slow and costly. Most of these calls end in a multiple-choice decision, such as which tool to call next. Decision models such as Jev answer these questions directly, at a fraction of an LLM call's time and price. We ask how a harness can delegate these decisions to a decision model with little loss of success. To evaluate decision models in agentic tasks, we build AgentDecisionBench, which tests them from isolated questions up to control of a running agent, on public benchmarks and our DecisionGym environment. The decision model most accurate on isolated questions solved only 69.5% of agent tasks, and without a fitted confidence threshold, delegating tool choices lowered success by 19.0 points. Building on these findings, we then design JevAgentHarness, which fits this threshold for each choice on the agent's own trajectories and lets the LLM decide below it. Because existing decision models never see the states an agent reaches after its own mistakes, we train AgentDecision-4B, an open 4B model, on such states with Jev's answers as labels. On tasks unlike its training data, the agent solved 91.5% of them with AgentDecision-4B and 82.0% with Jev. On tasks like their training data, 4B models trained this way made agents 81.1-89.0% cheaper and up to 2.5 times faster, without losing 3 or more points of task success. The models, data, and harness will be released at https://github.com/JevAgentHarness/JevAgentHarness.