AI 嘴上懂行规，手上全违规
专业伦理守则管着医生、律师、审计师，而他们正越来越多地把活儿交给大模型。这篇研究造了一个测试集：把 15 部俄语职业守则的每一条禁令，改写成 3,433 个「违规请求」，再配上只差一个合法前提的「孪生请求」，看 AI 会不会照做。结果：14 个主流模型组合里，11 个在大多数违规请求上直接产出违规内容，而且大多不吭一声；就算警告，四分之三的警告后面跟着的也是违规输出。更微妙的是，守规矩的模型靠的是「拒绝违规、但照常服务合法请求」，而开启推理模式能让两个模型的拒绝率减半——也就是说，想让它更守规矩，反而可能让它更会钻空子。这不是你明天能用上的工具，但它给了一个此前没人量过的数：AI 不是不懂行规，是懂了也不当回事。
📄 原文摘要(英文)
Codes of professional ethics guide and restrict specialists, who increasingly work with assistants built on large language models. Such an assistant can breach the user's code, or help the user breach it, without the user noticing. Evaluations reporting broad categories of harm cannot show how often this happens or which provision is breached. TenetBench pairs 3,433 requests conflicting with clauses of fifteen Russian-language professional codes with benign twins that differ only in the legitimacy of what is asked. An automated pipeline generates the tasks from the text of a code, so the corpus grows with every code added and a provision yields new tasks on every run. It also produced tasks and human-validated verdicts from two English codes of other jurisdictions. In human validation of the frozen pipeline, specialists rated 0.995 of the conflicting tasks plausible, and the model judge agreed with the adjudicated label at Gwet's AC1 of at least 0.894 on 1,860 elements. We evaluated fourteen model and reasoning-mode combinations on whether they produce the conflicting text, decline or warn. Eleven of the fourteen produce conflicting content on most such requests, mostly without a warning, and nearly three warnings in four accompany conflicting content. The models that conform more do so by declining while still serving the legitimate twin, and enabling reasoning halved the refusals of two models. We hope the dataset and pipeline will help evaluate how language models observe professional norms and let diagnostic datasets be built for particular professions and organisations.