OpenAI developed an internal red-teaming large language model called GPT-Red to probe and improve the defenses of its other models. The company released the latest version of its flagship model, GPT-5.6, last week and says training that model against GPT-Red made this release its most robust to date.
What GPT-Red is and why it was created
GPT-Red automates a form of safety evaluation known as red-teaming, a process traditionally performed by human testers to discover as many ways as possible to break or hijack a system so vulnerabilities can be fixed prior to release.
As LLMs grow more capable and are deployed as agent-like systems that can interact with files, websites, third-party code and other agents, human teams alone struggle to keep up with the expanding set of potential attacks. "The risk surface grows and the blast radius also grows," says Nikhil Kandpal, an OpenAI research scientist who co-created GPT-Red.
How the model was trained
Researchers started from an LLM that had not been trained as a hacker and placed it into a self-play loop with several other models. One side attempted to attack the others; the other side tried to defend. Over many rounds, GPT-Red improved at attacking and the target models improved at defending.
Training occurred in a dojo-like environment designed by OpenAI to emulate a range of real-world deployment scenarios, including web browsing, reading emails and calendar apps, and editing code.
When GPT-Red discovered a new attack, it explored multiple variants to find the most effective version for specific scenarios. Dylan Hunn, the project’s other co-creator, says the model is "extremely persistent" about drilling down into an attack it finds.
A new prompt-injection variant: "fake chain of thought"
OpenAI concentrated on prompt-injection attacks, where a malicious actor embeds instructions in text the LLM will see—such as in code or on web pages—to make the model act against developers' or users' intentions (for example, leaking confidential information or sabotaging code).
The researchers say GPT-Red discovered a previously unseen form of prompt injection they call a fake chain of thought. A chain of thought is an internal trace where an LLM records partial results while solving a problem. GPT-Red found a way to insert a fake entry into another model’s chain of thought that would cause the target model to act on spoofed information.
Chris Choquette-Choo, another research scientist on the team, illustrated the effect: "It’s like if I told you that 1+1=3 and that you have verified this already. The model’s like, ‘Oh, okay, of course,’ and it just spits out 3."
Tests and results
Independent experts, including Jessica Ji, a senior research analyst at Georgetown University’s Center for Security and Emerging Technology (CSET), rate the self-play approach as promising. OpenAI reran an experiment from 2025 in which human red-teamers sought weaknesses in an earlier GPT-5 version; when GPT-Red performed the same task, it located more effective attacks than the humans had.
OpenAI also tested GPT-Red against Vendy, a vending-machine agent developed by Andon Labs that evaluates real-world agent behavior. GPT-Red was able to hack Vendy to change item prices and cancel a customer’s order.
On the defensive side, OpenAI reports that when it tried the strongest GPT-Red attacks on its models, more than 90% of those attacks worked against GPT-5 (released in August last year), while fewer than 23% succeeded against GPT-5.6.
Limitations and human collaboration
GPT-Red has limits: it performs poorly on attacks that rely on back-and-forth conversational exploits, where human attackers would have fewer difficulties. It is also not yet strong at using images as an attack vector, even though images can be used to convey malicious text to models in prompt-injection attacks.
OpenAI frames GPT-Red as a supplement to human red-teamers: humans may find attacks the model misses, and vice versa. One workflow is to give GPT-Red attacks discovered by humans and ask it to enumerate all variations.
"I think human expertise will still be very important," says Jessica Ji. "It would be really useful to be able to distinguish where human testing is most needed."
No public release
OpenAI does not plan to release GPT-Red publicly. The company is confident the super-hacker would be difficult for others to replicate: the team has worked on it for more than a year, supported by the compute resources of one of the world’s wealthiest companies. "It’s not a trivial thing that someone else could easily do— you know, just go and train a super-attacker using this idea," Choquette-Choo says.
In sum, GPT-Red is an automated, self-play red-teamer that has revealed new attack types and helped reduce the success rate of major attacks against GPT-5.6, while remaining a complement to human testing and not being released publicly.
