Researchers at the University of Toronto warn that slowing down artificial intelligence remains an unsolved puzzle. Raymond Douglas, coauthor of a new report titled Pacing the Frontier, A Research Agenda, says the community does not yet understand what options exist or what their consequences would be.
Concerns have intensified after an Anthropic researcher left the company and warned that AI could wipe out humanity within a couple of years. The head of Anthropic’s AI safety lab echoed those concerns. Leaders at major American firms, including Dario Amodei of Anthropic, Sam Altman of OpenAI, Elon Musk of SpaceXAI, and Demis Hassabis of Google DeepMind, have all supported some form of slowdown or pause.
The urgency stems from companies using AI to build ever more powerful models. This has sparked fears of an accelerating recursive self-improvement loop where AI would outstrip human ability to comprehend its actions within a few years.
Independent evaluators
One idea is to give third-party evaluators greater access to models. These groups test capabilities and red team models by trying to elicit misbehavior in trusted environments.
Geoffrey Irving, former chief scientist at the UK AI Security Institute, believes rigorous inspections could effectively pause frontier AI development for now. “In the near term, inspections and audits work, or even just mutual agreements,” Irving says. “I do think the companies are afraid of RSI and misaligned takeoff.”
Some critics argue inspections must be more independent and scientifically rigorous than they currently are. The fact that some AI agents have recently escaped containment during testing suggests more rigor is required.
Connor Leahy, head of Control AI, a nonprofit that advocates for AI controls, says inspections should involve the FBI or the NSA. “When [big AI companies] say ‘independent evaluators,’ they mean ‘I want to pay my friends who live in my group houses to look at my prompts,'” Leahy says.
Douglas says new research could improve model evaluations. He points to recent work showing how outsiders can examine usage of models without disclosing any confidential information. Other techniques include new ways of peering inside AI models to get a better sense of what they are doing.
Leahy agrees there is a need for more research on model evaluation as well as what it actually means to align a model, or make it reflect human values, in the first place. “There has been a very deliberate marketing campaign from these companies to try to present evaluations as scientific,” he says. “But we don’t actually understand how AI works.”
How much the US government is willing to step in to restrict AI development is uncertain. President Trump has largely dismissed the need to regulate the industry, but there are signs that bipartisan support is growing for reigning in big AI.
Trusted compute
Some experts believe limits on AI development should ultimately involve checks on the raw compute required. The most powerful models are trained using thousands of Nvidia GPUs inside vast data centers.
The government has dabbled with tracking this already, through a 2023 Biden-era AI executive order that required companies to report training runs above a certain compute threshold.
A policy white paper from March 2024 argues that cloud providers could be crucial to future efforts because of their visibility into major AI training runs. The white paper suggests that tracking billing records, GPU utilisation, network traffic, and power consumption could provide proxies for AI capabilities.
Experts have also proposed ways of tracking and controlling efforts to build advanced AI by modifying chips themselves.
One idea, put forward by researchers at RAND in 2024, would involve modifying an existing component on GPUs used to measure performance so that it performs a cryptographically secured record of compute runs that can be inspected periodically. This could reveal, for example, that a company has been training AI above a certain threshold.
Others have suggested building new kinds of tamper-proof components into chips so that they collect detailed information about usage. These components would be required to run certain models’ weights using cryptography.
Some have even proposed building embedded off switches into chips so that they require remote cryptographic authorization to run certain models. They say this could prevent unauthorised parties from training AI models or deactivate chips if they fall into the wrong hands.
Binding treaties
Most experts agree that finding new ways to collaborate internationally will be crucial for controlling AI development, since other nations—especially China—also have the capacity to build frontier AI. “In the medium term, the simplest way is to unwind the hardware growth mutually with China, via a treaty,” Irving suggests.
The US has also sought to limit the development of Chinese AI by banning the exports of Nvidia’s most powerful chips. This has had limited success because companies can still train models using cloud compute from abroad.
The US and China are likely to discuss the risks of AI when President Xi visits the US later this month. While Chinese experts are also worried about the risks posed by rapidly advancing AI, they’re skeptical of a slowdown that would keep Chinese companies behind their US counterparts.
Some ideas for collaboration seem ripped from the pages of sci-fi rather than policy proposals.
Toby Ord, a philosopher at Oxford University specializing in existential risk, has previously mused that if nations can agree to limit the development of AI—and if the risk seems grave enough—then big nations might bring GPUs to a neutral territory and destroy them. Such a dramatic move would involve both countries agreeing to stop developing AI completely.
The entire puzzle is likely to be complicated further by recent technical progress, and the uncertainty around recursive self-improvement means that it will be especially important to track progress in that area.
A new benchmark called RSI Index is one of several new efforts to do that. Developed by Vals AI, a startup, the benchmark tries to track progress of AI-powered AI development by measuring the performance of public AI models against research published by human AI scientists. Rayan Krishnan, cofounder and CEO of Vals AI, says the benchmark suggests that within the next year AI could perform work that AI researchers cannot follow.
How best to keep an eye on AI isn’t just a technical challenge, though. The report from Douglas and others warns that rushing to implement inappropriate controls could result in the effort becoming mired in politics or subject to regulatory capture.
“I’m not sure if just telling the US government to shut it all down is going to end well,” Douglas says. “Going off half-cocked with a bad plan could end up worse than nothing.”
What it means
For people building models, the shift means moving from blind speed to measured verification. Companies like Anthropic now report using AI for 26 percent of their research and spending 6 percent of their compute budget on safety. The practical change is that progress will be audited before it is released, and hardware access may be restricted by external checks rather than internal choice.
