OpenAI Introduces GPT-Red, an AI Attacker Built to Strengthen GPT-5.6

OpenAI has revealed GPT-Red, an internal artificial intelligence system trained to attack the company’s own models, expose their vulnerabilities, and generate adversarial data that can be used to strengthen future releases.

Rather than functioning as another public chatbot, GPT-Red operates as an automated red teamer. It repeatedly sends malicious or misleading instructions to target models, observes their responses, and adjusts its strategy until it either succeeds or exhausts the available attack path. OpenAI says the system is particularly focused on prompt injection, a growing security problem for AI agents that interact with emails, websites, files, code repositories, and connected applications.

The system has already influenced OpenAI’s production models. Precursors to GPT-Red have been used during training since GPT-5.3, while the completed model was incorporated into the adversarial training of GPT-5.6 Sol. OpenAI reports that GPT-5.6 Sol produces six times fewer failures on its most difficult direct prompt injection benchmark than the company’s leading production model from four months earlier.

Why Prompt Injection Is Becoming More Dangerous

Prompt injection occurs when an attacker places instructions inside data that an AI system is expected to read. A malicious command might be hidden in an email, webpage, uploaded document, tool response, or software repository.

The AI agent can mistakenly treat that external content as a legitimate instruction. Instead of completing the user’s original task, it might disclose confidential information, upload files to an attacker-controlled server, change account settings, execute unsafe code, or take unauthorized actions through connected tools.

The problem becomes more significant as AI systems move beyond answering questions and begin acting on behalf of users. An agent with access to cloud storage, payment systems, source code, business applications, or internal company data presents a larger potential “blast radius” when its instructions are successfully manipulated.

OpenAI describes prompt injection as one of the central challenges created by agentic AI. Connected tools make models considerably more useful, but every new data source also provides another channel through which an attacker can attempt to influence the model’s behavior.

GPT-Red Learns Through Self-Play

GPT-Red was developed through self-play reinforcement learning, an approach in which an attacking model and a collection of defender models improve by competing against one another.

The attacker receives a reward when it produces a valid failure, such as convincing a model to follow a prompt injection. The defender is rewarded for rejecting the malicious instruction while still completing the legitimate task it was originally given.

As the defenders become better at recognizing attacks, GPT-Red must find more sophisticated and diverse ways to bypass them. Those new attacks can then be incorporated into the defenders’ training, creating an escalating cycle between offensive and defensive capabilities.

OpenAI built a large collection of simulated environments for this process. Depending on the scenario, GPT-Red might control part of a local file, the text of an email, a webpage banner, or the output returned by an external tool. Each environment defines what the attacker can manipulate and what outcome qualifies as a successful compromise.

The company says GPT-Red was trained using computing resources comparable to some of its largest post-training runs, representing an unusually large allocation of compute specifically for safety research. By the end of training, the model could compromise nearly every internal and production model it was tested against up to and including GPT-5.5.

GPT-Red Outperformed Human Attackers in Testing

To evaluate whether GPT-Red could generalize beyond its training environments, OpenAI tested it using a replicated version of a 2025 indirect prompt injection challenge.

GPT-Red and human red teamers were asked to develop attacks against GPT-5.1 in scenarios that had not been included in GPT-Red’s training. The automated system produced successful attacks in 84% of the scenarios, compared with 13% for the human participants.

That result does not mean human security researchers have become unnecessary. GPT-Red’s advantage lies largely in its ability to conduct a high volume of experiments, discard unsuccessful strategies, and continually refine promising attacks without the time and staffing constraints of a human team.

Human testers remain important for identifying unexpected risks, understanding the wider consequences of failures, and designing scenarios that automated systems may not consider. Jessica Ji, a senior research analyst at Georgetown University’s Center for Security and Emerging Technology, described the results as promising while emphasizing that human expertise remains essential.

A Vending Machine Became a Real-World Target

OpenAI also tested whether attacks discovered in simulations could transfer to functioning AI agents.

One experiment targeted an autonomous vending machine agent created by Andon Labs and deployed in an OpenAI office. GPT-Red was initially given a simulated version of the system, along with the ability to observe how the agent used its tools.

After testing and refining its attacks in the simulation, GPT-Red transferred them to the production agent. It achieved all three assigned objectives: lowering the price of an expensive product to $0.50, ordering a new item worth more than $100 and offering it for $0.50, and cancelling another customer’s order. OpenAI says the vulnerabilities were disclosed and additional safeguards are being tested.

The experiment illustrates why prompt injection is not simply a matter of producing an inappropriate chatbot response. When an AI model controls external systems, a successful injection can result in financial transactions, modified records, data loss, or physical-world consequences.

GPT-Red was also tested against a Codex command-line agent powered by GPT-5.4 mini. Across 10 previously unseen data-exfiltration scenarios, it was more successful and more token-efficient than a prompted GPT-5.5 baseline at convincing the coding agent to expose sensitive information.

How GPT-Red Changed GPT-5.6

One of GPT-Red’s most notable discoveries was a category of direct prompt injection that OpenAI calls a “fake chain-of-thought” attack.

At a high level, the technique presents fabricated reasoning or verification information that encourages the target model to assume a malicious instruction has already been reviewed and approved. OpenAI researcher Chris Choquette-Choo compared the concept to falsely telling someone that an incorrect answer had already been verified, making them more likely to accept it without repeating the underlying reasoning.

These attacks reportedly succeeded more than 95% of the time against GPT-5.1. After adversarial training using attacks generated by GPT-Red, the success rate fell below 10% against GPT-5.6 Sol. Across a broader set of direct prompt injection environments, OpenAI says GPT-5.6 Sol failed on only 0.05% of GPT-Red’s attempts.

OpenAI also evaluated whether the added defenses simply made GPT-5.6 more likely to refuse legitimate requests. The company says general capabilities and targeted over-refusal evaluations remained largely unaffected, suggesting that the improvements came from better differentiation between malicious and legitimate instructions rather than from making the model broadly less willing to act.

That distinction is important. A model can appear secure if it refuses to open files, browse websites, execute code, or interact with external applications, but it would also lose much of its practical value. Effective robustness requires preserving legitimate tool use while resisting instructions inserted by untrusted parties.

Why OpenAI Is Not Releasing GPT-Red

GPT-Red will remain an internal system and is being kept separate from OpenAI’s publicly deployed models.

That decision reflects the dual-use nature of automated red teaming. The same model that can help developers discover prompt injection vulnerabilities could also help attackers develop more effective methods for compromising AI agents operated by other companies.

OpenAI’s approach is to retain the attacker internally while transferring the resulting defensive knowledge into production models. The company says this separation is intended to prevent the malicious capabilities deliberately trained into GPT-Red from becoming accessible to outside adversaries.

This also means GPT-Red should not be confused with OpenAI’s public cybersecurity models or defensive programs. It is not a penetration-testing product, nor is it primarily designed to autonomously discover conventional software exploits. Its current focus is attacking the instruction-following behavior of AI systems, particularly agents exposed to potentially untrusted data.

Automated Red Teaming Still Has Blind Spots

Despite its strong results, GPT-Red does not provide a complete solution to AI security.

Reporting on the research indicates that the system remains less effective at multi-turn attacks that unfold gradually across a long conversation. It also has limited capabilities for developing prompt injections embedded in images, leaving important multimodal attack surfaces insufficiently covered.

The results are also based heavily on environments and success criteria designed by OpenAI. Although the company tested GPT-Red on held-out scenarios and functioning agents, independent researchers will have limited ability to reproduce the findings while the model and much of its evaluation infrastructure remain private.

OpenAI says it will continue combining automated testing with human and third-party red teaming, layered product safeguards, access controls, monitoring, and real-time abuse detection. That defense-in-depth strategy reflects the reality that no single model or benchmark can anticipate every attack that might emerge after deployment.

A New Security Race Between AI Attackers and Defenders

The “self-improvement” described in OpenAI’s announcement is not a deployed model autonomously rewriting its own weights or independently creating a more powerful successor. Instead, it is a controlled training loop in which one AI system generates adversarial examples that researchers use to improve another.

Even so, GPT-Red represents a meaningful shift in how frontier models may be secured. Human red teams can uncover sophisticated vulnerabilities, but they cannot manually generate the scale and variety of attacks required to continuously train increasingly capable AI agents.

Automated attackers can fill part of that gap, creating a security flywheel in which each stronger defender forces the red-teaming model to discover new weaknesses. Those weaknesses then become training material for the next generation of production systems.

The risk is that similar capabilities will not remain exclusive to defensive laboratories. As open and commercial models become better at long-horizon planning, tool use, cybersecurity, and autonomous experimentation, attackers will gain access to increasingly capable systems of their own.

GPT-Red therefore marks both progress and an early indication of the contest ahead. AI laboratories are beginning to use powerful models to defend their next generation of agents, but those defenses will need to evolve continuously as the same underlying technologies make automated attacks cheaper, faster, and more adaptable.