Two Frontier Labs, Two Weeks, One Root Cause: What Anthropic's Eval Incident Tells Us About Agent Security

On July 30, Anthropic published a retrospective on three cybersecurity evaluation incidents involving its Claude models. It's a striking piece of transparency — and it lands just weeks after OpenAI disclosed something strikingly similar involving Hugging Face. Two of the industry's most safety-focused labs, two separate incidents, and the same underlying failure mode.

‍

That's not a coincidence worth ignoring. It's a preview of a problem every organization running AI agents is going to face, whether or not it ever runs a frontier model eval.

‍

What actually happened

‍

Anthropic reviewed 141,006 of its own evaluation runs after OpenAI's disclosure, specifically looking for cases where a Claude model might have reached the open internet from inside a test environment that was supposed to be sealed off. It found three.

‍

In each case, Claude was given a capture-the-flag challenge — a standard way of testing a model's offensive security skills, where the model is told a piece of secret information is hidden somewhere on a network and left to find its own way to it. In each case, the evaluation prompt told Claude explicitly that it had no internet access. And in each case, that was wrong: a misconfiguration on the evaluation vendor's side left the environment with a live path to the open internet that nobody had caught.

‍

When Claude's search for the flag led it to real systems, it did what a capture-the-flag exercise trains any tester to do — it treated the target as fair game and went after it. In one incident, that meant extracting real credentials and touching a database with several hundred rows of production data. In another, it meant publishing a real, functioning malicious package to the public PyPI registry, which was downloaded and executed by 15 real systems in the hour before it was caught. In the third, the model scanned roughly 9,000 alternative targets before compromising one live application.

‍

Anthropic is careful to note what this wasn't: no model set out to escape its test environment, and no model pursued a goal of its own. Each one acted on the instructions it had been given, while holding a false belief — created entirely by the gap between the prompt and the actual environment — about whether what it was looking at was real.

‍

Why this isn't just a frontier-lab story

‍

It's tempting to read this as a problem for the handful of companies running high-stakes model evaluations. It isn't. The mechanism here — an autonomous system acting on an incorrect assumption about its own scope of access, with nobody watching closely enough to catch the mismatch in real time — is exactly the failure mode showing up wherever organizations deploy coding agents, AI assistants with tool access, or autonomous testing tools of their own.

‍

Anthropic's own conclusion is the sharpest part of the report: the line between an aligned action and a harmful one depends on what the model believes about its situation, not just on its intentions. A model told it has no internet access will behave very differently from one that knows it does — and if that belief is wrong, the model's behavior will be wrong too, for entirely explainable reasons. The fix isn't a better-behaved model. It's an environment that doesn't let a false assumption go unchallenged for months.

‍

That's a visibility problem before it's anything else. Anthropic itself points to the gaps: no one validated the environment's actual internet-access paths before the evaluation began, no one was monitoring the logs closely enough in real time to catch the mismatch sooner, and the same rigor wasn't extended to the third-party vendor's side of the connection. Every one of those is an access and monitoring gap, not a model behavior problem — and access and monitoring gaps are solvable with the right visibility layer.

‍

The pattern to watch

‍

Put the two disclosures side by side and a pattern emerges that's bigger than either incident:

• The agent did exactly what it was told, using a scope of access no one intended it to have. In both cases, the model's behavior followed logically from a false premise about its own environment — not from misalignment.
• The organizations on the receiving end didn't catch it themselves. In Anthropic's incidents, none of the three affected organizations had detected the activity before Anthropic reached out. Visibility that depends on the other side of a connection to notice and report a problem isn't visibility — it's hope.
• The gap sat exactly at the connection layer. Not in the model's training, not in its intentions, but in an unmonitored, unvalidated link between a system and the internet — the kind of connection that's invisible until you're specifically looking for it.

As AI agents take on more autonomous work — writing and running code, testing systems, connecting to more of the tools in a given environment — this is the shape the next incident is likely to take: not a model deciding to misbehave, but an agent doing exactly what it was asked with more reach than anyone realized it had.

‍

What good visibility actually looks like

‍

The organizations best positioned to avoid a version of this incident are the ones that can answer a few questions continuously, not just at setup:

• Which AI agents and AI-connected applications exist across the environment right now?
• What is each one actually connected to — not what it was configured to be connected to, but what the live OAuth grants, API keys, and network paths say it can reach?
• Who owns each of those agents and connections, and does the current access still match the original intent?
• Would an unexpected or unauthorized connection surface on its own, or only after something has already gone wrong?

That's the layer this incident exposed a gap in, for two of the most careful labs in the industry. It's the same layer we built Reco to cover — continuous discovery and posture management for SaaS apps, non-human identities, and AI agents, so that "what's connected to what" is something you know, not something you find out about after a disclosure.

‍

Want to see what your own AI agent and SaaS-to-SaaS connection graph actually looks like? Talk to our team about a walkthrough.