The UK Government’s Mythos AI Tests Add Skepticism to Cybersecurity Hype
Last week, Anthropic announced it was restricting the initial release of its Mythos Preview model to “a limited group of critical industry partners,” giving them time to prepare for a model that is “strikingly capable at computer security tasks.” The UK government’s AI Security Institute (AISI) has now published an initial evaluation of the model’s cyberattack capabilities, adding some independent public verification to Anthropic’s reports.
- The AISI’s findings show that Mythos isn’t significantly different from other recent frontier models in tests of individual cybersecurity-related tasks. However, Mythos could set itself apart through its ability to effectively chain these tasks into the multistep series necessary for full system infiltration.
- AISI has been evaluating various AI models since early 2023, with performance rising steadily from GPT-3.5 Turbo’s inability to complete low-level “Apprentice” tasks to Mythos Preview’s completion of north of 85 percent of those same tasks.
- The UK government’s tests suggest that while Mythos is impressive in its individual cybersecurity capabilities, it may not be significantly more effective than other recent models at the complex multistep attacks needed for full system infiltration. This adds a layer of skepticism to the initial hype surrounding the model.
Originally published at arstechnica.com. Curated by AI Maestro.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.