AI Red Teaming Breakthrough
Reported by MarkTechPost: OpenAI Details GPT-Red: An Internal Automated Red-Teaming Model That Beat Human Red-Teamers 84% To 13% On Prompt Injection. Analysis and context written by TickrWire.
Evolving story · 3 updatesOpenAI GPT-Red safety systemTimeline →OpenAI's GPT-Red model outperformed human red-teamers in a prompt injection test, with a success rate of 84% compared to 13%. The model was trained using self-play reinforcement learning against a population of defender LLMs.
- GPT-Red outperformed human red-teamers in a prompt injection test
- The model was trained using self-play reinforcement learning
- GPT-Red discovered a novel attack class called 'Fake Chain-of-Thought'
- The model helped reduce the failure rate of GPT-5.6 Sol by a factor of six
OpenAI has developed an internal automated red-teaming model called GPT-Red, which has demonstrated impressive capabilities in identifying vulnerabilities in language models. The model was trained using self-play reinforcement learning, where it played against a population of defender LLMs. This approach allowed GPT-Red to learn and improve its attack strategies.
The results of the test were striking, with GPT-Red outperforming human red-teamers by a significant margin. The model's success rate of 84% compared to the human team's 13% highlights the potential of automated red-teaming in AI security testing.
One of the key findings of the test was the discovery of a novel attack class called "Fake Chain-of-Thought". This attack exploits the way language models process and generate text, and it has significant implications for the development of more secure AI systems.
OpenAI has also reported that GPT-Red has helped to reduce the failure rate of its GPT-5.6 Sol model by a factor of six on a direct injection benchmark. However, the company acknowledges that there is still work to be done, particularly in addressing multi-turn and image-based attacks.
The development of GPT-Red is an important step forward in the field of AI security, and it has significant implications for the development of more secure and robust language models.
Improved AI security testing
More secure language models
Potential for increased investment in AI security
Advancements in AI security
- red-teaming
- The practice of testing a system's defenses by simulating an attack
AI ResearchAI Method Reveals What Genomic Models Learn From DNA and Exposes Hidden Experimental Bias
AI ResearchPew study confirms sharp rise of AI-written text on the web since ChatGPT's launch
AI ResearchKids outlearn AI—and we still don’t know why
AI ResearchWho’s behind the new ‘stealth model’ Ox Alpha?
AI ResearchHarvey Introduces Harvey Tenet: A Kimi K3 Base Post-Trained with Fireworks for Long-Horizon Legal Agent Work
BusinessHow AI-native companies turn workflows into operating capability
OpenAI highlights how firms like Basis, Clay, and Exa Labs deploy autonomous agents to handle onboarding, account management, and developer integrations.
SecurityPath to Astra: critical capabilities and frontier safeguards
OpenAI’s Astra model is the first to meet the Critical cybersecurity capability threshold under its Preparedness Framework, enabling autonomous discovery of unknown vulnerabilities and exploit chains.
BusinessHow law firm Gilbert + Tobin governs and scales AI with OpenAI
Gilbert + Tobin, a major Australian law firm, has rolled out ChatGPT Enterprise and Codex firm-wide, driven by CEO Sam Nickless and supported by rigorous governance. The adoption rate, with 87% of enabled seats active, more than doubles the firm's typical tool usage, and specific workflows such as recruitment research have been cut from four hours to about 20 minutes.
AI ToolsPolimill builds Japan's next-generation public AI infrastructure
Polimill introduced QommonsAI, an OpenAI‑powered platform that now supports roughly 1,050 Japanese local governments and 550,000 public employees, aiming to become a shared operating system for municipal work.
BusinessA milestone in expanding access to AI
OpenAI announced that its ChatGPT Ads platform has crossed $1 billion in annualized revenue run rate in under 200 days and is rolling out self-service tools internationally.
FundingIndia’s Ringg gets backing from Peak XV as it pushes voice AI past the phone call
Indian voice AI startup Ringg has raised $10 million in a Series A extension led by Peak XV Partners, bringing its total funding in the round to $15.5 million.