AI Pulse
📄 论文解读

大模型隐藏的思考过程,被一个加密漏洞扒光了

OpenAI、Anthropic 这些大厂为了不让别人偷走自家模型的推理过程,把思考步骤加密后发给用户。但研究者发现,这些加密块在不同会话、不同用户、甚至不同模型之间是通用的——于是他们把这串密文塞给同一家公司的弱模型,弱模型会乖乖把它解密并原样念出来。用这招,他们从公开的代码仓库里捞了 31 万多个推理块,翻出 367 条个人信息和 182 组账号密码;更阴的是,还能把恶意指令藏在加密块里,让 AI 在不知不觉中被操纵。这不是你明天能拿来用的漏洞,但它说明一件事:AI 厂商以为加密能藏住的东西,在架构层面可能早就漏风了。

📄 原文摘要(英文)

Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.

arXiv 原文

订阅 AI Pulse

每天 08:00 · 12:30 · 18:30 · 23:50 更新