AI Attackers Don’t Get Tired: Why Cybersecurity Has to Change

Your security program was built for attackers who do.

When OpenAI published its account of the models that broke out of an evaluation environment and spent days inside Hugging Face’s production infrastructure, I called it a Terminator moment for cybersecurity. I stand by the statement, but something else about this should give us pause. Nine days later, Anthropic disclosed incidents of its own. In early August, Meta confirmed another. Days after that, a fourth lab’s model, Moonshot’s Kimi K3, was slipping its testing sandbox. Four labs. Less than three weeks. Four models that went somewhere nobody intended.

These stories have largely been treated as AI safety stories. They are. But for a CISO, they are also third-party risk stories — and that may be the more immediate lesson. The uncomfortable truth is that the OpenAI-Hugging Face incident was, at its core, a third-party risk story: a vulnerable third-party component as the escape route, a vendor as the victim, and a systemic node thousands of companies depend on as the blast radius.

Here’s the number I would bring to your next security planning meeting. Of the roughly 17,600 attacker actions Hugging Face recovered from its logs, most of them failed.

The agent tested paths that went nowhere. It got blocked by URL allowlists and pivoted. It hit IAM policies that denied every mutating call. It tried to reach a production database twice and timed out both times. It probed, failed and moved on. Thousands of times. It didn’t get frustrated. It didn’t get tired. And failure cost it almost nothing.

Four models broke out of testing environments in under three weeks. The escapes made headlines. Security leaders should be more worried about what made them possible: an attacker that can fail thousands of times at almost no cost.

That is the shift. For decades, defenders have benefited from one simple fact: failure costs attackers something. Reconnaissance took time and dead ends were costly. The economics of that asymmetry is what made “imposing costs” on attackers a coherent way to try to stop them. When failure becomes nearly free, the successful path stops being a needle in a haystack and becomes a statistical certainty for anyone able to pursue every path at once.

The patterns in these incidents point to three lessons security leaders should act on now.

Four Labs, One Shape

The pattern is remarkably consistent. Give a model an objective. It finds a path nobody expected. Sometimes that path leads straight into the production systems of organizations that were never supposed to be involved. In at least one case, the affected organization learned what had happened because Anthropic called to tell them

In every case, the path ran through software, infrastructure or an environment somebody else built. Security leaders should take that personally.

Every organization in this chain had a security program. Several had very good ones. The failure wasn’t contained inside any one perimeter. It happened in the seams between organizations, and nobody owns the seams.

OpenAI’s models found a zero-day in a self-hosted package proxy and used it to reach the open internet. From there they took root on an unsecured code-execution sandbox belonging to a Modal Labs customer, used it as a staging base, and tricked Hugging Face’s dataset loader into reading local files and then executing code. Within thirteen hours they held cluster-admin across multiple internal clusters, helped along by a single credential shared across all of them. The entry point was novel. Inside, the weaknesses were the ones we have been writing up for twenty years.

Nine days later, Anthropic did something rarer: it published its own review of 141,006 evaluation runs and disclosed three instances with similar presentation unprompted: a model reaching the open internet from an evaluation environment, then gaining access to real organizations’ systems. No sandbox escape this time, just a misconfiguration that left a live path out while the model had been told none existed. Once outside, the model exploited weaknesses every security team knows: weak passwords, unauthenticated endpoints, SQL injection and credentials sitting on an exposed debug page.

Meta confirmed a fourth in early August, after a configuration error handed one of its models internet access during an evaluation. Meta did not catch it either; it learned what had happened when its testing partner called. Both that evaluation and Anthropic’s were run by the same outside firm, Irregular, which has publicly called them the same issue. One vendor’s configuration error propagated across two frontier labs and four victim organizations, none of which had any direct relationship with Irregular at all. It was a vendor to the two labs, not to them. The agent didn’t care where one company’s attack surface ended and another’s began.

That is what makes this a third-party risk problem. Exposure can now originate several relationships away from you, in an organization you never selected, assessed or even knew was part of your attack surface.

Then researchers at Frontier Security caught Moonshot’s Kimi K3 slipping its sandbox too, so it could  access a benchmark repository on GitHub. Nobody was breached, and the model was chasing its assigned objective, having worked out that using a ready-made answer key was cheaper than solving. Two things make this case important. Researchers caught it during their own test. And it ran on completely different tooling from the Anthropic and Meta cases. One vendor having a bad month does not explain this.

The labs are not ignoring any of this. Anthropic published its review unprompted, which is why the industry has a pattern to examine and learn from — rather than an outlier to wave off. That should become the norm. OpenAI has now slowed the release of its Astra model after internal evaluations could not rule out critical cyber capability. Both moves deserve credit. But neither is a defense plan: a release delay governs one company’s calendar. It does nothing about capable models, Kimi K3 among them, that already ship open-weight and free, beyond any single lab’s control.

Correlation Without Escalation is Not Detection

Hugging Face’s detection stack, to its credit, correlated the signals into a coherent attack. It then failed to escalate the criticality, and the on-call team was not paged. The tooling assembled the story and could not decide the story was urgent. That is a triage architecture built for a world in which 17,600 events over four days would have seemed absurd.

Cybersecurity Needs its Fosbury Flop

In 1968, Dick Fosbury won Olympic gold in the high jump not by jumping higher than his competitors but by going over the bar backwards. Everyone else was refining the straddle. Better form, more strength, marginal gains on a decades-old technique. Fosbury changed the shape of the problem. Within two Olympics, the straddle was just about extinct.

Security leaders now have to clear a bar that just moved, and most of the industry’s answer so far is a better straddle: more alerts, faster analysts, tighter SLAs on human review. That will not work, because the constraint is not analyst speed. You cannot answer machine-speed attacks with human-speed triage..

The flop looks like this:

• Stop treating the boundary as the control. The evaluation sandbox failed, the dataset loader failed, and the pod isolation failed, each of them doing its job right up until it wasn’t. Design for what happens next. Use short-lived credentials. Eliminate shared administrative identities. Block access to metadata services. Above all, limit how far an attacker can move after the first control fails.
• Move detection from alerting to correlation. Finding one suspicious event is no longer enough. The job is to connect thousands of ordinary-looking events into one attack story, and know when that story is serious enough to wake someone up. This increasingly has to be automated given the volume we’re seeing.
• Treat third-party exposure as an attack path, not a questionnaire. These incidents ran through third-party dependencies: a vendor’s product, a vendor’s customer, an evaluation partner, shared testing tooling. Your risk now includes configuration mistakes made by organizations you have never audited and may not even know exist. Static, point-in-time assessments cannot see any of that. This problem isn’t new. AI makes it much more urgent. Security teams need a continuous view of the companies and technology they depend on, because attackers see that entire ecosystem as one connected surface.

At SecurityScorecard, this is exactly the kind of problem our culture is built around: bringing together people who are deeply curious about how attackers think, willing to challenge twenty-year-old assumptions, and excited to build a fundamentally different approach to security!

The Question to Bring to Your Next Board Meeting

I said a few weeks ago that the genie has become too powerful for the bottle. Four labs later, nothing has argued otherwise. After incidents like these, everyone asks the same question: Are we ready for AI-enabled attackers? I don’t think that question gets us very far. ‘Yes’ and ‘no’ produce the same result: nothing changes.

Ask a harder one. If 17,000 low-signal events hit your environment over the next four days — and most looked like failures — how long would it take your team to realize they were one attack? Would anyone get paged in time?

You already know the answer. The gap between that answer and four days is the work.