Your AI Security Tool Isn’t Protecting You – It’s Just Making You Feel Better

In June 2025, Microsoft patched CVE-2025-32711, better known as EchoLeak: a zero-click vulnerability in Microsoft 365 Copilot, scored 9.3 on CVSS. A crafted email, never opened by the victim, could make Copilot’s retrieval engine quietly exfiltrate whatever sensitive context it had access to. The detail that should bother you: the payload walked straight past XPIA, Microsoft’s own prompt-injection classifier, which was present, running, and working exactly as designed.

Bruce Schneier gave us the vocabulary for this back in 2009. Security theater, he wrote, describes measures that make people feel more secure without doing anything to improve their security. The feeling and the reality are different things, and they diverge.

Seventeen years later, AI has industrialized the divergence, on both sides of the equation. AI-powered security tools are sold on capabilities they can’t demonstrate under measurement, and security tools for AI are sold as solutions to a problem their own documentation admits is unsolved. Gartner’s 2025 hype cycle placed AI trust, risk and security management at the peak of inflated expectations. The analysts are telling you where we are. The question is what to do about it.

The gap between the datasheet and the measurement

Start with what happens when someone tests deployed controls instead of reading their marketing.

The Picus Blue Report 2025 analyzed more than 160 million real-world attack simulations run against production environments in the first half of 2025. Picus sells the simulation platform, so weight the source accordingly, but the numbers are hard to dismiss. Prevention effectiveness averaged 62%, down from 69% a year earlier. Despite logging coverage of 54%, only 14% of simulated attacks generated an alert. Attacks using valid credentials succeeded 98% of the time. And of the data-exfiltration attempts simulated, the controls in place blocked 3%.

Three percent. These are environments with modern, frequently AI-badged security stacks, and the exfiltration test is the one that maps most directly to what an attacker wants to do.

The benchmark scores that appear in vendor decks deserve the same skepticism. When vendors began claiming perfect marks on the MITRE ATT&CK evaluations, Forrester analyst Allie Mellen published a pointed reminder: the evaluations have no winners or losers, and the 100% claims typically rest on unrealistic configurations, cherry-picked sub-results, or detection settings that would bury a real SOC in noise.

None of this is new, which is the depressing part. In 2019, researchers at Skylight Cyber took Cylance’s “pure AI” antivirus apart by appending strings from the video game Rocket League to malware samples, flipping the classifier’s verdict on 100% of the top ten malware families of the day and 83% of a broader sample set. The lesson, that static machine-learning models fail against adversaries who study them, was published seven years ago. The market’s response was to ship more static machine-learning models.

The guardrail problem is worse

If AI-powered detection oversells, the tools sold to secure AI itself are in a stranger position: the standards body that defines the threat says the threat may not be fixable.

Prompt injection sits at number one in the OWASP Top 10 for LLM applications, and OWASP’s own guidance is unusually blunt: given the stochastic nature of the models, “it is unclear if there are fool-proof methods of prevention for prompt injection.” Fine-tuning and RAG, it adds, do not fully mitigate it. That’s the problem the prompt-shield market exists to solve, described as possibly unsolvable by the organization that catalogued it.

The empirical work backs the pessimism. Researchers at Lancaster University and Mindgard tested six production guardrail systems, including Microsoft’s Azure Prompt Shield and Meta’s Prompt Guard, and achieved up to 100% evasion success using character-injection and adversarial perturbation techniques, while keeping the underlying attacks functional. HiddenLayer went further, publishing a single transferable “Policy Puppetry” prompt template it says bypasses every major model on the market. That one is vendor research and not peer-reviewed, so treat the “all” with care, but independent outlets reproduced the core claims.

The most credible witness has no product to sell. The UK’s AI Safety Institute, evaluating five leading LLMs, reported that “all models were highly vulnerable to our basic attacks”, with every model complying at least once in five attempts for almost every harmful question when in-house attacks were applied. Basic attacks. Not nation-state tradecraft. A government evaluation body, politely noting that the emperor’s input filter has no clothes.

Unite.AI’s own coverage has catalogued the vulnerability classes and the injection techniques for years. None of this is secret. It just doesn’t appear on datasheets.

The dangerous part is the feeling

Here’s where the title of this article stops being rhetorical. If these tools merely underdelivered, the loss would be budgetary. The research suggests something worse: the confidence they generate actively degrades behavior.

Safety researchers call it risk compensation, or the Peltzman effect: protected people take more risks. Drivers with seatbelts drive faster. And organizations with AI security dashboards, it turns out, stop doing the boring things.

The numbers line up uncomfortably well. Cisco’s 2025 Cybersecurity Readiness Index, surveying 8,000 security leaders, found that 86% of organizations had experienced an AI-related security incident in the previous twelve months, while only 10% considered AI the hardest thing they have to protect, and 60% admitted they lack visibility into how employees use generative AI at all. Incidents near-universal; concern, a rounding error.

IBM’s 2025 Cost of a Data Breach report completes the picture. Of the organizations that suffered breaches of AI models or applications, 97% lacked proper AI access controls. One in five breaches traced back to shadow AI, adding roughly $670,000 to the average breach cost, and 63% of breached organizations had no AI governance policy at all. Read those numbers together and a pattern emerges: the failures are boring. Absent fundamentals, in organizations that felt covered.

The dashboard didn’t malfunction. It delivered its actual product, which was confidence.

What the honest version looks like

None of this argues that AI security tooling is worthless, and the best counter-evidence deserves airtime. Anthropic’s Constitutional Classifiers survived more than 3,000 hours of red-teaming by 183 researchers without a universal jailbreak, cutting jailbreak success from 86% on the undefended model to 4.4%, at the cost of a 0.38-point rise in over-refusals and roughly 24% compute overhead. Then Anthropic ran a public challenge, and out of 339 participants, four cleared every level and one found a working universal jailbreak anyway.

That’s the honest shape of the whole field in one result: a well-built guardrail raised the attacker’s cost enormously, carried a measurable tax, and still fell to a sufficiently determined human. If I had to compress this article into a purchasing principle, it’s this: a control sold as a cost-raiser with published failure modes is worth evaluating; a control sold as a solved problem is selling you the feeling.

Control or comfort blanket: three questions

You can separate one from the other without a research budget. Three questions, asked of every AI security tool you own or are about to buy.

What is its measured bypass rate in my environment? Not the vendor’s benchmark. Yours. Continuous validation and purple-team exercises exist precisely because, as the Picus data shows, deployed controls drift toward failure silently. A number you haven’t measured is a number you’re taking on faith.

What happens when it fails? Not if. OWASP’s practical guidance points the same direction as every incident above: least-privilege access for models, human approval gates on sensitive actions, and segmentation that assumes the filter will eventually let something through. EchoLeak was survivable for organizations whose Copilot simply couldn’t reach anything worth stealing.

Is the vendor’s claim a fact or a hypothesis? Even the vendors concede more than their datasheets do. Vectra’s own survey of 2,000 SOC professionals, vendor research again, found teams able to handle just 38% of the alerts their tools generate, with 60% saying vendors sell tools that create noise rather than clarity, and half calling their tooling more hindrance than help. When the industry’s own customers report that, “trust the datasheet” stops being a strategy.

The pattern outlives the patch

EchoLeak was fixed in a Patch Tuesday. The pattern it demonstrated, a production guardrail confidently filtering the front door while the attack came through the mail slot, was not, and the research record above says it won’t be soon.

Schneier’s distinction is seventeen years old and has never been more expensive to ignore. The feeling of security is a product, and the AI era sells it in subscription tiers. The reality of security is a measurement, and nobody can sell it to you, because you have to go and take it.