OpenAI’s new GPT-6 Astra model blocks 99.99 per cent of direct prompt injection attacks but still fails to prevent hidden instructions from altering its output in roughly one out of twelve scenarios.
Performance gains and remaining risks
The system produces fewer factual errors than its predecessor, GPT-5.6 Sol. OpenAI tested the new model against ChatGPT conversations that users had flagged for wrong answers. These were particularly error-prone cases, meaning the failure rates should not be taken as typical for everyday use. Astra reproduced these reported errors much less often. The biggest improvements appeared at low latency settings and lower reasoning levels.
For direct prompt injections, where users try to manipulate the model through their own prompts, Astra hits a near-perfect 99.99 per cent defense rate. OpenAI credits its GPT-Red method for this. The technique uses an automated attacker to harden the model during training.
Jailbreak resistance looks similar. Against a fixed dataset of known attacks trying to extract harmful responses about biology, violence, and cybersecurity, Astra refuses to help in 91.5 to 98.3 per cent of cases.
When attackers adapt their strategy over multiple conversation rounds, Astra’s defense rate drops to about 67 per cent. This means persistent adversaries can coax out at least one problematic response roughly one in three tries. Predecessor models scored just under 50 per cent on the same test. OpenAI notes that these tests ran on the bare model without the production safety layers like classifiers that ship with the actual product.
Indirect attacks remain a threat
Astra makes progress on indirect prompt injections, where an attack is buried inside a document the AI reads. External testing by security firm Gray Swan, using 1,810 curated attacks from their IPI Arena, found that with 15 attempts per scenario, Astra was cracked at least once 8.5 per cent of the time. GPT-5.6 Sol failed 27 per cent of the time. Claude Opus 5 did better at 4.8 per cent in the same evaluation, but it was not immune either.
The numbers in Gray Swan’s combined Q1 and Q2 test actually went up compared to earlier results. Anthropic previously reported only a two per cent attack success rate based on the easier Q1 test alone, and GPT-5.6 Sol scored just 20 per cent there too. Anthropic also ran all models with extended reasoning turned on. This, alongside the broader test scope, could explain the gap.
Even though these are curated, hand-picked attacks, the success rates should worry any enterprise security team. Astra can be tricked through injected instructions in roughly one out of every twelve scenarios. Opus 5 holds up better, but it still fails about one in twenty-one. And the risk is growing. AI agents are increasingly writing code, operating tools, and controlling computers on their own, which is what Gray Swan tested. These agents are also being built to run around the clock and at scale, with reading and processing documents as one of their core jobs.
What it means
Developers building autonomous agents face a clear trade-off. The new model is safer against obvious user manipulation and factual errors, which helps with general reliability. However, the vulnerability to hidden instructions in documents means that agents processing untrusted files remain a security liability. Enterprises cannot yet assume that simply switching to the latest model eliminates the risk of an agent executing malicious code embedded in a document it is asked to read.
