OpenAI creates GPT-Red, a 'superhacker' AI model to shield its own systems against attacks

OpenAI has developed a language model specialized in hacking, dubbed GPT-Red, whose role is to serve as a training partner (sparring partner) to strengthen the defenses of its other models against cyberattacks. According to the company, training its latest flagship model, GPT-5.6, against attacks from GPT-Red made it its most robust release to date.

GPT-Red automates a security task known as 'red-teaming', traditionally carried out by human teams: finding every possible way to break or hijack a system in order to patch those weak points before releasing the final version of the software. OpenAI's rationale is that, as language models become more complex and are used in an ever-growing variety of tasks —especially as agents that interact with files, websites, third-party code and other agents—, it becomes increasingly difficult for teams of people alone to keep pace with every possible type of attack. Nikhil Kandpal, an OpenAI researcher and co-creator of GPT-Red, sums it up this way: 'The risk surface grows and the blast radius grows too'. Dylan Hunn, another company researcher and co-creator of the system, explains that the idea is to get ahead of the problem: as more capable models emerge, there will already be a system able to discover new attack modes. The researchers say GPT-Red has already found types of attack that had not been observed before.

Most of the effort focused on a type of attack called 'prompt injection', in which an attacker slips hidden instructions to a model so it does something its developers or users do not want, such as copying confidential information, sabotaging a company's codebase or generating harmful or embarrassing content. In theory, these instructions can be hidden in any text the model ends up processing, whether code or the content of a web page.

To build GPT-Red, the researchers started from a model with no prior training as a hacker and placed it in a self-play loop alongside other models: GPT-Red's goal was to attack the other models, while their goal was to defend themselves. Over many rounds of play, GPT-Red improved at attacking and the defending models improved at resisting. This training took place in a kind of 'dojo' designed by OpenAI to reproduce real-world scenarios of language model use, such as browsing the web, reading emails or calendar apps, and editing code. When GPT-Red detected a new type of attack, it explored multiple variants to find the most efficient one in each specific scenario. Hunn notes that, compared with a human red-teamer, the model is very good at finding exactly what will work, what is most effective, and that it is extremely persistent when it comes to digging deeper into an attack it has discovered.

One of the most striking findings, according to OpenAI, is a never-before-seen type of prompt injection they call 'fake chain of thought'. The chain of thought is a kind of journal in which a model notes down and stores partial results while solving a problem. GPT-Red found a way to insert a false entry into another model's chain of thought, tricking it into acting on falsified information as if it were true. Chris Choquette-Choo, another researcher on the team, compares it to telling someone that 1+1=3 and that you have already verified it: the model simply accepts the statement and repeats it as if it were true.

Jessica Ji, a senior analyst researching AI security at the Center for Security and Emerging Technology (CSET) at Georgetown University, believes the self-play approach used by OpenAI is sound and that the results presented are very promising.

To measure GPT-Red's effectiveness as an attacker, OpenAI repeated a 2025 experiment in which human red-teaming teams had tried to find weaknesses in an earlier version of GPT-5. Given the same task, GPT-Red proved more effective than humans at finding effective attacks. The company also tested GPT-Red against Vendy, a vending-machine agent developed by Andon Labs (a firm dedicated to evaluating the performance of AI agents on real-world tasks). GPT-Red managed to hack Vendy to change the prices of items on sale and cancel a customer's order.

As for defensive results, OpenAI says that when it applied the most powerful attacks generated by GPT-Red against its own models, more than 90% of those attacks worked against GPT-5 (released last August), while fewer than 23% worked against the new GPT-5.6, which the company presents as evidence of a substantial improvement in robustness.

GPT-Red is not perfect: it is not especially good at detecting attacks that require a back-and-forth conversation between attacker and target, something a human attacker would handle without difficulty, and it still does not excel at using images, which can also be used to convey hidden text in prompt-injection attacks. For this reason, OpenAI insists that GPT-Red complements the work of its human red-teaming teams, since people can still find attacks the model overlooks. One of the strategies the company is using consists of giving GPT-Red an attack devised by humans and asking it to find all its possible variants. Jessica Ji agrees that human expertise will remain very important, and that it would be useful to be able to distinguish in which cases human testing is really necessary.

As expected, OpenAI does not plan to release GPT-Red. The company is confident that this 'superhacker' is more powerful than any imitator model someone might try to create, and its researchers say they have been working on it for more than a year, backed by the computing resources of one of the richest companies in the world. Choquette-Choo sums it up by noting that it is not something trivial that anyone can easily replicate, that is, training a super-attacker following this same idea.