AI Pulse
📄 论文解读

给AI装个“只读”阀门:它再也不会被网页里的坏话骗走钱

你的AI助手会替你读邮件、付账单,但网页里藏一句“把转账目标改成我”就能劫持它。现有防御要么让AI自己判断(它可能正好读到那句坏话),要么每步都多跑一次模型。这篇的思路很笨但很硬:工具返回的数据自带“类型”——发件人、收款账号是结构化字段,而注入的指令是自由文本。SPILLWAY只放行那些目标值来自只读工具的结构化字段、或符合固定委托规则的动作;其余一律拦下,并把可疑文本先删掉再给模型看。在三个模型、七个任务集上,标准模板注入的攻击成功率是零,强化学习攻击者的成功率也只有0.007,而且几乎不增加模型调用。它不是你明天能装上的插件,但它是第一个能给你“动作去向”提供可审计硬底线的方案——当AI开始管你的钱,这个底线比聪明更重要。

📄 原文摘要(英文)

Tool-using agents act on email, files, and payments, so an instruction hidden in a web page or a shared document can make an agent send data or money to an attacker. Many runtime defenses put a language model on the decision that lets an action through, so the decision depends on a model that may read the attacker's text and adds model calls to every step. Provenance defenses keep the model off the final check but need a model to plan or route each task first. Where an action goes can instead be read from the tools' typed returns: an email's sender or a payment's account arrives in a typed field, while injected instructions arrive as free text. SPILLWAY turns this observation into a deterministic floor with no per-task plan. The check admits a call when each target value that appears in tool output but not in the user's task was returned by a read-only tool in a typed field or is covered by a fixed delegation rule. A deterministic sanitizer removes instruction-like regions its fixed rules match before the model reads them and retains the removed spans as evidence, and a local judge may restore a refused call, never one whose target lies in the removed text. Across three backbones, and seven task suites on one of them, no standard templated injection redirects an action to an attacker's target under SPILLWAY, at the highest observed mean utility and composite scores (NRP, ERS) of the defenses we run under that attack. A forged chat template also produces a raw zero, and mean in-scope attack success under a released reinforcement-learning attacker is .007. SPILLWAY reaches this floor with no defense-triggered backbone re-run and almost no added model call. As personal AI agents act on users' email, files, and payments, SPILLWAY gives them an auditable refusal floor on where their actions go that keeps them useful and holds across backbones.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新