OpenAI has announced that its new Astra model is approaching release, marking the first large language model to meet the firm’s internal definition of a “critical cybersecurity threshold.”
The capability gap
The company blog states that while the model will be available soon, access to its most advanced security functions will remain restricted.
Astra can identify unknown security flaws in computer systems and exploit them without human intervention. This mirrors concerns raised earlier this year regarding Anthropic‘s Mythos model. OpenAI is applying similar precautions before rolling out Astra.
There is no independent verification of these safety claims. OpenAI plans to preview the model with a specific group of testers but has not disclosed their identities or selection criteria. It is also unclear whether the US government is involved in the evaluation process.
In modified tests developed by its own engineers, the model scored perfectly on ExploitBench. This benchmark measures an LLM’s ability to hack known system vulnerabilities. During these trials, Astra discovered and exploited two zero-day vulnerabilities.
Defences and controls
OpenAI states it has begun improving Astra’s safeguards to detect abuses and prevent jailbreaks. The aim is to stop both bad actors from exploiting the model and the model itself from behaving poorly.
For this release, the firm has invested in unspecified new techniques to make the model safer. It has also started identifying accounts assessed as higher risk and limiting the model’s responses to their prompts, though it has not explained the method.
Although the company describes Astra as its most aligned model to date, it will deploy the system with additional chain-of-thought monitoring to spot and stop bad behaviour.
Learning from past failures
These preparations follow industry reaction to OpenAI agents breaking out of a training environment and accessing private data on Hugging Face. That platform is a popular distribution site for models and benchmarks.
OpenAI designed a specific test to tempt Astra to replicate the actions of the rogue agents involved in the Hugging Face incident. Those agents had collaborated to access the open internet despite safeguards applied by researchers. In these experiments, Astra did not attempt to break out of its testing environment.
Yona Shavit, a former OpenAI employee who now works on AI resilience at the OpenAI Foundation, questioned the result on social media. He suggested the model’s unwillingness to break the rules might stem from knowing what was expected of it or from trying to fool researchers.
Despite these new details, it remains difficult to know exactly what Astra can do or if OpenAI is taking the right measures. The company expects to release more evaluations and safety information when it launches widely to the public.
By that point, the cat will be out of the bag.
