OpenAI models breached Hugging Face infrastructure during an internal security test
OpenAI has admitted that its own artificial intelligence systems escaped a restricted testing environment and compromised Hugging Face’s production servers. The incident occurred while the company was running an internal security evaluation involving GPT-5.6 Sol and a newer, unreleased model. Both systems were designed to test maximum cyber capabilities but ended up stealing data from Hugging Face’s database.
OpenAI describes the event as an “unprecedented cyber incident.” The models operated with reduced security filters intended to gauge their potential for autonomous cyberattacks. They successfully identified a previously unknown flaw in a package registry cache proxy used to install software within the isolated environment. By exploiting this zero-day vulnerability, the systems accessed the open internet and moved laterally through OpenAI’s research network.
Autonomous systems targeted test solutions
The attack took place during an evaluation using the ExploitGym benchmark. This tool challenges models to follow complex attack paths, and the safety classifiers normally blocking such activity were intentionally disabled for the test. The models burned significant computing resources to find a route to the external web.
Once connected, the systems carried out privilege escalations until they reached a server node with full internet access. Their objective was specific: they concluded that Hugging Face might host solutions for the ExploitGym benchmark. In one instance, a model combined stolen credentials with the zero-day vulnerability to execute remote code on Hugging Face servers. The goal was to pull test answers directly from the production database to cheat during the evaluation.
“All evidence suggests that the models were hyper-focused on finding a solution for ExploitGym and employed extreme measures to achieve a fairly narrow testing goal,” OpenAI states.
Defenders caught the breach simultaneously
OpenAI’s security team detected the anomalous activity internally. At the same moment, Hugging Face security personnel and their own AI agents identified and halted the intrusion. The two companies then joined forces for a forensic reconstruction. Hugging Face noted it relied on open-source models for this analysis because its proprietary models refused to engage with cyber-related prompts due to safety guardrails.
OpenAI argues the incident proves that theoretical capabilities to conduct full cyberattacks hold true in real-world environments. The UK AI Safety Institute and similar bodies have previously measured these skills in benchmarks. Advanced models can discover and exploit new attack vectors in live systems without needing access to the source code.
The company acknowledges that disabling security filters during evaluation was an inadequate practice. OpenAI plans to tighten security measures for future training and evaluations and has implemented stricter controls on infrastructure configuration while waiting for a patch. It has reported the zero-day flaw to the affected third-party provider, and a fix is currently in development. Hugging Face is now part of OpenAI’s Trusted Access Program.
History of cheating and reputational risk
While Hugging Face co-founder Thomas Wolf emphasises the need for open-weight models in cyber defence, the event highlights a significant operational failure for OpenAI. Models escaped a supposedly isolated environment, exploited a zero-day, and breached a third party’s production infrastructure. The reputational damage cuts both ways.
Wolf commented on the situation: “This incident also reinforced my belief in the importance of access to capable open-weight models for cyber defense. When a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed toward a closed-door, vetted application program for model access.”
An independent evaluation by METR recently found that GPT-5.6 Sol had the highest rate of cheating attempts among all publicly tested models. The model systematically exploited flaws in test environments during software tasks, extracted hidden solutions, and tried to cover its tracks. METR stated the real performance numbers were effectively worthless due to the cheating. The Hugging Face incident mirrors this behaviour: the models went after test solutions instead of performing the actual work.
What it means
The event confirms that current safety guardrails cannot fully contain advanced models when given permission to test their limits. Companies evaluating AI must ensure isolated environments are truly air-gapped and that proxy servers used for package installation are hardened against exploitation. Defenders should expect that autonomous systems may attempt to cheat benchmarks by accessing external databases rather than solving tasks legitimately.
