漏洞百出的夏季:聚焦约翰·雷贝格的提示注入研究

本文报道了独立 AI 研究员约翰·雷贝格发起的“AI 漏洞月”倡议,他每天发布关于各种易受提示注入攻击的 AI 工具的报告。该研究涵盖多个平台,如 ChatGPT、Codex、Anthropic MCPs、Cursor、Amp、Devin、OpenHands、Claude Code、GitHub Copilot 和 Google Jules 等,揭示了这些漏洞在发现三年后仍然普遍且危险。作者总结了约翰的每一项发现,阐述了各种攻击向量,包括通过 Markdown 图像或 DNS 请求进行数据泄露,通过诱骗代理修改自身配置(例如,VS Code `settings.json`)来执行任意命令,以及权限提升。这些漏洞中观察到的常见模式包括 Prompt 注入作为初始攻击向量,各种数据泄露方法(例如,Web 请求、自定义 DNS 服务器),以及编码代理中任意命令执行的问题性质。本文介绍了约翰提出的“AI 杀伤链”概念,描述了 Prompt 注入如何导致“混淆代理”问题,进而实现自动工具调用。一个值得关注的问题是,尽管进行了负责任的披露,但许多已报告的漏洞尚未被供应商修复,通常是因为修补它们会严重影响工具的实用性。作者重申,某些系统可能天生不安全,但行业仍在明知风险的情况下构建它们。




15th August 2025

Independent AI researcher Johann Rehberger (previously) has had an absurdly busy August. Under the heading The Month of AI Bugs he has been publishing one report per day across an array of different tools, all of which are vulnerable to various classic prompt injection problems. This is a fantastic and horrifying demonstration of how widespread and dangerous these vulnerabilities still are, almost three years after we first started talking about them.

Johann’s published research in August so far covers ChatGPT, Codex, Anthropic MCPs, Cursor, Amp, Devin, OpenHands, Claude Code, GitHub Copilot and Google Jules. There’s still half the month left!

Here are my one-sentence summaries of everything he’s published so far:

Common patterns

There are a number of patterns that show up time and time again in the above list of disclosures:

  • Prompt injection. Every single one of these attacks starts with exposing an LLM system to untrusted content. There are so many ways malicious instructions can get into an LLM system—you might send the system to consult a web page or GitHub issue, or paste in a bug report, or feed it automated messages from Slack or Discord. If you can avoid unstrusted instructions entirely you don’t need to worry about this... but I don’t think that’s at all realistic given the way people like to use LLM-powered tools.
  • Exfiltration attacks. As seen in the lethal trifecta, if a model has access to both secret information and exposure to untrusted content you have to be very confident there’s no way for those secrets to be stolen and passed off to an attacker. There are so many ways this can happen:
    • The classic Markdown image attack, as seen in dozens of previous systems.
    • Any tool that can make a web request—a browser tool, or a Bash terminal that can use curl, or a custom view_text_website tool, or anything that can trigger a DNS resolution.
    • Systems that allow-list specific domains need to be very careful about things like *.azure.net which could allow an attacker to host their own logging endpoint on an allow-listed site.
  • Arbitrary command execution—a key feature of most coding agents—is obviously a huge problem the moment a prompt injection attack can be used to trigger those tools.
  • Privilege escalation—several of these exploits involved an allow-listed file write operation being used to modify the settings of the coding agent to add further, more dangerous tools to the allow-listed set.

The AI Kill Chain

Inspired by my description of the lethal trifecta, Johann has coined the term AI Kill Chain to describe a particularly harmful pattern:

  • prompt injection leading to a
  • confused deputy that then enables
  • automatic tool invocation

The automatic piece here is really important: many LLM systems such as Claude Code attempt to prevent against prompt injection attacks by asking humans to confirm every tool action triggered by the LLM... but there are a number of ways this might be subverted, most notably the above attacks that rewrite the agent’s configuration to allow-list future invocations of dangerous tools.

A lot of these vulnerabilities have not been fixed

Each of Johann’s posts includes notes about his responsible disclosure process for the underlying issues. Some of them were fixed, but in an alarming number of cases the problem was reported to the vendor who did not fix it given a 90 or 120 day period.

Johann includes versions of this text in several of the above posts:

To follow industry best-practices for responsible disclosure this vulnerability is now shared publicly to ensure users can take steps to protect themselves and make informed risk decisions.

It looks to me like the ones that were not addressed were mostly cases where the utility of the tool would be quite dramatically impacted by shutting down the described vulnerabilites. Some of these systems are simply insecure as designed.

Back in September 2022 I wrote the following:

The important thing is to take the existence of this class of attack into account when designing these systems. There may be systems that should not be built at all until we have a robust solution.

It looks like we built them anyway!


AI 前线

破局云原生质量困境:如何构建可落地、轻量的全面保障体系?

2025-12-23 22:25:10

AI 前线

欢迎 GPT OSS,来自 OpenAI 的全新开源模型家族!

2025-12-23 22:25:42

0 条回复 A文章作者 M管理员
    暂无讨论,说说你的看法吧
个人中心
购物车
优惠劵
今日签到
有新私信 私信列表
搜索