AI Pulse
📄 论文解读

AI 拼图式泄密:单看每份文件没事,合起来就出事

你让 AI 助手去查几份机密文件,它可能把每份里看似无害的信息拼在一起,直接说出一个不该说的结论——这就是「马赛克效应」。研究者用 30 个医疗、军事、网络安全的合成案例测了 3 个 AI 智能体,发现即使你问的是「单份文件就能回答」的普通问题,最强模型也有 7% 的概率泄露禁止的复合结论;在真实运行环境里更糟。给文件加上「机密」标签、明确禁止拼凑规则,能把泄露率压下来,但仍有 32% 的答案在违规。更麻烦的是:AI 拒绝回答的次数变多,反而可能暴露它已经猜到了那个敏感结论。这不是你明天能修好的 bug,而是 AI 在混合权限环境里工作时的固有风险——它提醒我们,光靠「别告诉 AI 太多」不够,得重新设计 AI 能推断和说出的边界。

📄 原文摘要(英文)

Modern LLM agents can retrieve information from accessible documents, and then disclose a sensitive conclusion assembled from them. This mosaic effect creates a critical risk for agents operating over confidential data. We quantify its realization with 30 synthetic cybersecurity, medical, and military examples, yielding 10,800 final answers from three LLM agents in controlled simulation and 1,440 answers from live agents in two open-source harnesses. Even under benign queries designed to be answerable from one document, GPT-5.6-Sol, the best performing model tested, discloses forbidden composites in 7.08% of answers (95% CI: 3.75-10.67%) in simulation, with worse disclosure rates in live harnesses. Adding visible classification headers and explicit aggregation rules reduces pooled disclosure rates substantially, yet unsafe disclosures persist in 32.37% of simulated answers (95% CI: 28.33-36.19%) across all intents despite these safeguards. Furthermore, increased refusals do not guarantee confidentiality, since significant refusals reveal the prohibited conclusion regardless. These results point to a troubling conclusion: sensitive data leaks via mosaic-style events are an observed confidentiality failure mode in the evaluated settings at non-negligible rates. They motivate stronger controls on what agents may infer and disclose in sensitive data environments with mixed-clearance users.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新