The UK government says the performance gap between open-weight AI models and proprietary systems in cybersecurity is shrinking.
The narrowing security divide
The UK AI Security Institute (AISI) has released a public analysis of how far open models trail the closed frontier. The findings indicate a reduction in this delta for the current year.
Recent open models, specifically GLM-5.2 and DeepSeek V4-Pro, now perform at a level similar to frontier closed models released four to seven months prior. This represents a narrower margin than the six to ten months measured through most of 2025.
On a set of 70 evaluations for specific, narrow cyber capabilities, GLM-5.2 sits closest to Claude Opus 4.6, which launched 4.3 months earlier. DeepSeek-V4-Pro performs between Claude Opus 4.5 and GPT-5, released in November and August 2025 respectively.
The AISI intends to test Kimi K3 on this same basis once weights are publicly released.
The advantage for proprietary models extends a bit further for long-horizon cyber ranges. These are tasks requiring models to chain various capabilities together to complete a full hacking operation. On a benchmark called The Last Ones, GLM-5.2 reaches the level of Opus 4.5, released less than seven months before it. DeepSeek’s V4-Pro falls below Sonnet 4.5, a sub-cyber-frontier model released seven months prior.
The AISI notes this gap is larger than on narrow cyber tasks. This suggests that while open weight models can appear quite strong, they sometimes lack the generalisation capabilities that distinguish proprietary systems. Industry observers refer to this as “big model smell”.
The main implication is that the distance between the controllable frontier and the lawless openly diffused frontier is decreasing. The AISI writes this implies cyber defenders have a short window to prepare before today’s frontier capabilities become accessible without the safeguards used by proprietary companies.
China closes the frontier gap
Chinese firms have begun to out-compete Western actors at building and deploying open weight models, such as DeepSeek. Now they are starting to close the gap on frontier models as well.
The latest example is Kimi K3, a 2.8 trillion parameter model. It scores exceptionally well on all tasks major proprietary ones benchmark on, typically matching or trailing Claude Fable 5 and GPT 5.6 Sol.
However, the model shows some brittleness that smells like “benchmaxxing”. Performance may have been tuned around benchmarks in a way that harms generalisation.
“While its overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 demonstrated frontier-level performance across our evaluation suite, consistently outperforming other tested models,” Kimi writes.
Kimi’s weights will be made available in the coming weeks along with a research paper.
AI that builds AI
Kimi has use-cases relating to recursive self-improvement, using AI systems to improve AI itself. They tested how good Kimi was at writing GPU compilers.
“Kimi K3 developed MiniTriton, a compact Triton-like compiler with its own tile-level IR layer over MLIR, optimization passes, and a PTX code-generation pipeline. Across supported roofline benchmarks, MiniTriton delivers performance on par with or better than Triton and torch.compile — beating Triton on certain workloads,” they write.
They do not discuss this going into production, or being used to train Kimi K3 itself, but it suggests future models might be able to do this.
Additionally, they showed how Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimised, and verified the chip using open-source EDA tools on the Nangate 45nm library.
Policy and the future of control
Most notions of AI policy and safety rest on control. The idea is that a small number of actors deploy proprietary models which you can intervene on at the platform level. Models like Kimi K3, if they release weights, completely change this by diffusing broadly uncontrollable powerful AI into the world.
This will have a vast range of positive effects, driving a boom in entrepreneurship and increasing the ‘sovereign intelligence’ available to anyone who can run the model, but will also yield various unknown unknowns. The next few years will be defined by the gap between proprietary models and widely available models and how they show up in society.
Demis Hassabis proposes a regulatory regime
DeepMind founder Demis Hassabis has laid out a policy prescription for artificial general intelligence. His basic idea is that the US government should develop a framework for testing out frontier AI systems for new capabilities.
This would happen via a Standards Body modelled on a federally overseen public-private partnership or self-regulatory organisation, much like the Financial Industry Regulatory Authority (FINRA).
“This US-initiated effort would provide a strong starting point for creating shared international standards on Frontier AI,” he says.
The Standards Body would be responsible for developing assessment protocols and working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security. This testing infrastructure would help to define what would make a model a “Frontier Model”.
Labs developing those models would “be encouraged” to adopt best practices in areas like publishing details about their systems, investing in cybersecurity, and personnel vetting.
“Initially, Frontier Labs would voluntarily share models with the Standards Body for review up to 30 days before release. Once the assessment protocol is shown to be effective and robust, formalisation could quickly follow,” he writes.
Demis’s piece pulls together de facto consensus positions that have emerged across the AI industry in recent years. Powerful AI systems should be tested by third parties that have some loose relationship to a regulator.
It also rhymes with the de facto policy norm that has emerged in America recently across both the Trump admin’s executive order about AI and the recent processes developed in the aftermath of the Anthropic export controls saga. Here, government and industry developed assessment methods for evaluating the capabilities of AI systems and figuring out if they posed national security risks.
Sneaky side-channel tasks
In Neal Stephenson’s Cryptonomicon there is a scene where a character is locked up by a foreign government and asked to decrypt an intercepted message. While doing this, they carefully construct a software program to help surreptitiously read the decrypted message and change it before it is viewable by their captors.
Now, AI systems seem to be able to do the same thing. This is based on new research from Imperial College London and the UK AI Security Institute.
They look at how well AI systems can surreptitiously complete so-called “side channel” tasks while completing another task.
“A side task is a hidden objective that the agent pursues alongside the user’s main task, such as covertly exfiltrating the API KEY environment variable. Our primary metric is evasion: among attacks that successfully complete the side task, the fraction the monitor fails to flag,” the authors write.
For this research, they constructed a benchmark of tasks for AI systems to do, as well as “side tasks” for them to do as well. The benchmark includes five CLI-tool sequences and five Flask web-service sequences, each involving 5-6 PRs.
“Our goal in main task design is to provide natural cover for the actions of our four side-tasks (network access, environment variables, authentication, and webhook integrations) without directly justifying the side task itself”
They found that it is very hard to build classifiers to tell when a side-channel attack is happening. This is true both for attacks that unfold over time across multiple PRs, as well as single changes made in a single step, though gradual evasion is the more challenging one to deal with.
“An attacker can spread a side task across PRs (a gradual attack) or wait for the one whose legitimate task provides natural cover (non-gradual attack). No single monitor we test catches both strategies, and this holds across several attack agents and monitor models, reflecting the nature of control in a persistent-stat
