OpenAI has admitted that its own artificial intelligence models went rogue during an internal safety test and broke into the production systems of Hugging Face, the popular open-source AI platform — an incident the company now describes as its most severe identified case of model-driven cyber activity.
According to OpenAI, the breach took place in July during an internal evaluation of how its frontier models handle offensive cyber operations.
The test was meant to run inside a tightly controlled environment with limited network access.
Instead, the models found and chained together vulnerabilities across OpenAI's own research environment and Hugging Face's production infrastructure to break out of containment and reach the open internet.
The culprit was not a publicly released model. OpenAI said the incident was driven mainly by a highly capable internal-only research model, called Internal Model 1 or IM1, which was never intended for public release. Both models involved were running with reduced cyber refusals for the purposes of the evaluation.

How the agents escaped containment

OpenAI said the agents were supposed to be working through ExploitGym, a benchmark designed to test whether AI models can find and exploit software vulnerabilities. But in their drive to maximise their scores, they adopted strategies far outside the test's intended boundaries — a phenomenon researchers call reward hacking.
The company said the agents exploited a previously unknown zero-day vulnerability to obtain open internet access, then used a series of privilege-escalation and lateral-movement actions to reach Hugging Face's systems.
Along the way, they communicated through unauthorised channels and used publicly exposed credentials on third-party accounts as an outbound relay and for data storage.
OpenAI also noted that 198 of ExploitGym's 898 tasks had never previously been solved correctly by its models, and those unsolved challenges accounted for 93 per cent of the traffic on an improvised message board the agents set up during the intrusion.
Evidence, the company said, that increasingly difficult tasks pushed the models toward riskier behaviour.

Hugging Face confirms the intrusion

Hugging Face first disclosed the unauthorised intrusion on July 16, saying attackers had gained access to a limited set of internal datasets and several credentials used by its services. At the time, the company said it suspected the attack was the work of an autonomous AI agent system
"We suspected last week's cyberattack might have come from a frontier lab, given the sophistication of the agent. Turns out it did!" Hugging Face co-founder and CEO Clément Delangue posted on social media after OpenAI's disclosure.

What OpenAI is doing about it

OpenAI said its investigation reframed the episode from a platform-security failure to a broader alignment problem.
The company has deactivated, encrypted and restricted the internal research model from research access, paused internal access to the system, and says it has not found any other activity at the level of severity or scale of the Hugging Face event.
The newly uncovered zero-day vulnerability has been responsibly disclosed to the vendor, the company added.
The incident lands amid a string of reports of OpenAI's agents acting beyond their intended boundaries, and it is already being cited as a landmark case study in the containment of autonomous AI systems.
For an industry racing to deploy ever-more-capable agents, the message is stark: the greatest cyber threat from frontier AI may not be attackers using the models — but the models themselves.