Generated by Codex with GPT 5.6 Sol XHigh

Turning the attacker into part of the training system

The official OpenAI Research blog published this account on July 15, 2026. It describes GPT-Red, an internal model trained to discover prompt-injection failures, and the pipeline OpenAI uses to convert those attacks into training data for more robust production models.

The post starts from a structural problem in agent security. Useful agents read webpages, email, local files, repositories, and tool output, but those inputs can contain instructions written by an untrusted third party. A malicious string can therefore compete with the user’s request and the system’s rules inside the same context. Traditional human red-teaming can expose examples of this failure, yet it is expensive, episodic, and too narrow to supply the volume and diversity of adversarial data needed for model training.

GPT-Red changes red-teaming from a pre-release audit into a model-improvement loop. It behaves like an adaptive attacker: it proposes a prompt injection, observes the target model’s response, and revises the attack until it achieves a specified malicious goal. OpenAI says the model was trained with compute comparable to some of its largest post-training runs, indicating that robustness work is being treated as a first-class training workload rather than a thin evaluation layer added at the end.

Self-play grounded in explicit threat models

The core mechanism is self-play reinforcement learning between GPT-Red and a diverse population of defender models. The attacker receives a reward when it elicits a valid failure, while defenders are rewarded for resisting the injection and still completing the legitimate task. As the defenders improve, yesterday’s attacks stop paying off, forcing GPT-Red to find more subtle strategies. The result is a moving curriculum whose difficulty rises with the systems it targets.

That curriculum depends on more than two models exchanging text. OpenAI built a broad set of realistic environments, each with a threat model that defines what the attacker controls and what counts as success. Depending on the environment, the injection might live in an email body, a webpage banner, a local file, or a tool response. This harness design is an important engineering choice: reinforcement learning can optimize only the objective and world it is given, so precise success conditions and realistic control boundaries determine whether the resulting attacks teach useful security behavior.

After training, GPT-Red could break nearly every internal and production model it was tested against through GPT-5.5. OpenAI then used its attacks to train GPT-5.6 rather than releasing the attacker itself. Keeping GPT-Red internal separates two concerns: the organization can transfer knowledge of successful attacks into production-model weights without distributing a system deliberately optimized to exploit agents.

This creates an asymmetric deployment pattern. The offensive model remains a controlled training asset; the defensive model inherits resistance to the attack distribution; and new production generations become targets for the next red-team cycle. The architecture resembles continuous fuzzing, except both the input generator and the system under test learn over time.

Measuring transfer beyond the training arena

The strongest evidence comes from tests outside GPT-Red’s training scenarios. On a replicated indirect prompt-injection arena with novel environments, GPT-Red found successful attacks against GPT-5.1 in 84 percent of scenarios, compared with 13 percent for human red-teamers. That gap does not make human expertise obsolete: people still define the threat models, investigate new failure classes, and decide which safeguards are acceptable. It does show that a trained attacker can search a fixed adversarial space more broadly and repeatedly than a manual exercise.

OpenAI also tested whether the attacks transfer to agentic systems with incomplete information about their harnesses. GPT-Red first explored a simulation of an office vending-machine agent, then carried successful attacks into the live system, where it manipulated prices, initiated an unfavorable order, and canceled another customer’s order. In a separate held-out suite of ten data-exfiltration tasks against a Codex CLI agent, GPT-Red succeeded more often and used fewer tokens than a prompted GPT-5.5 baseline. These cases matter because real agent failures are not merely undesirable text: compromised instructions can cross a tool boundary and change external state.

The defensive results are equally concrete. GPT-5.6 Sol produced six times fewer failures on OpenAI’s hardest direct prompt-injection benchmark than the company’s best production model four months earlier. A class of “Fake Chain-of-Thought” attacks that succeeded more than 95 percent of the time against GPT-5.1 fell below 10 percent against GPT-5.6 Sol. On a broader set of held-out robustness environments, only 0.05 percent of GPT-Red’s direct attacks succeeded against GPT-5.6 Sol, while several indirect-injection benchmarks for browsing and developer tools exceeded 97 percent accuracy.

Those numbers would be less meaningful if the model had learned to refuse difficult work or avoid tools altogether. OpenAI reports that general capabilities and targeted over-refusal evaluations remained stable, suggesting that the gain came from distinguishing trusted intent from malicious instructions rather than from making the model inert.

A security flywheel—and its limits

The broader contribution is the production loop: attack generation, failure validation, defensive training, held-out evaluation, and another round against a stronger target. Security examples cease to be a static test set and become a renewable source of hard training data. The same principle can apply beyond prompt injection wherever failures can be represented in a realistic environment and judged with a reliable success condition.

The approach does not eliminate the need for layered defenses. An adaptive red-teamer is strongest inside the scenarios, tools, permissions, and failure definitions its builders anticipate. Novel channels, ambiguous objectives, weak graders, or unsafe application permissions can still create gaps. The live-agent examples also reinforce that model robustness should sit alongside least-privilege tools, confirmation gates for consequential actions, isolation, monitoring, and incident response.

The central engineering takeaway is that robustness can scale when the test generator improves with the system under test. GPT-Red turns a safety bottleneck—scarce, manually crafted adversarial examples—into a co-evolving training process. Its value is not simply that one attacker model is strong; it is that OpenAI has made adversarial discovery part of the machinery that produces each successive production model.