AI做选择题,终于学会说“我不会”
现在的AI做选择题,本质是把选项串成一段文字再猜答案,它分不清“没答案”和“答案不确定”,换个选项顺序还可能改判。这篇提出一个叫Jeva的架构,把“问题”和“每个选项”分开编码,再用对称聚合保证选项怎么排列结果都不变,还单独算一个“覆盖率”来区分“真没答案”和“选项里有竞争”。结果是:它几乎不会在选项换序时翻案,能处理上千个选项,而且重复出现的选项可以缓存复用、大幅降延迟。它不是你明天就能装上的东西,但“AI终于能诚实说不会”这件事,是决策类AI往前走的一步。
📄 原文摘要(英文)
System one models enable direct, low-latency decision-making. Yet recent practice of predicting an answer from serialized text does not, by itself, respect the structure of a decision problem. We argue that current designs underemphasize four important structural requirements. (i) null-aware: Models should distinguish answer absence from uncertainty among candidates; (ii) permutation-invariant: Reordering alternatives should not change the decision; (iii) option-scalable: Growing candidate sets should not require redefining the output space; (iv) cacheable: Request-independent computation over recurring alternatives should be reusable across requests. We introduce Jeva, a decision-native architecture that addresses these requirements jointly. A shared encoder independently represents the request and each option, while a compact decision stack refines only the request state against immutable option memory. Symmetric aggregation yields permutation-equivariant probabilities; shared scoring accommodates variable-size candidate sets. A separate coverage estimate distinguishes answer absence from candidate competition, enabling explicit null outcomes. Request-independent option representations and layer projections support reuse across newly assembled sets. These mechanisms unify Choice, Noul, and Score outputs. Across evaluations, Jeva combines strong typed-decision performance with near-perfect null detection and no decision flips under any tested permutations. It scales to thousands of candidates and substantially reduces latency through cross-request reuse without changing decisions.