What this means for makers and artists
If you build tools that generate sound, images, or code, the landscape is shifting from voluntary transparency to mandatory oversight. Anthropic CEO Dario Amodei argues that the era of self-regulation is over. He posits that unregulated AI is advancing so rapidly that it now functions as a strategic weapon for nation-states rather than just a consumer gadget. For creators, this signals a future where the models you train and the algorithms you deploy may face strict third-party audits before they ever reach the public.
The urgency of the speed problem
Amodei frames the central issue as a speed mismatch between technological progress and political response. He illustrates this with a subplot from “Lord of the Rings,” where two hobbits attempt to wake Treebeard to defend his forest against an army actively chopping it down. The dilemma is that Treebeard is wise but agonisingly slow; it takes him a full day just to greet another tree, making timely action impossible.
In Amodei’s view, the slow tree represents the political system, while the urgent hobbits are the early alarmists like Anthropic. The approaching army is the threat of unregulated AI, which he says is advancing with exponential speed based on scaling laws. He predicts that within one to two years, we could see “Powerful AI,” effectively “a country of geniuses in a data centre.”
Why transparency is no longer enough
Anthropic’s previous strategy relied heavily on transparency requirements, as the risks were previously too vague for precise regulation. The company supported legislation such as SB 53 in California, RAISE in New York, and SB 315 in Illinois.
Amodei now argues that this approach has failed. He cites the disruption caused by “Claude Mythos Preview,” which altered the global cybersecurity landscape and demonstrated that frontier models pose tangible risks to financial sectors, critical infrastructure, and national security. He warns that biological risks and serious autonomy failures could arrive even sooner.
Mandatory audits and the power to block
The new stance demands mandatory testing by qualified third parties across four specific risk areas: cybersecurity, biological weapons, loss of control over AI systems, and automated research and development that could accelerate those risks. Amodei suggests a government agency should have the authority to block or pull models deemed to pose unacceptable risk.
He points to the FAA as the model for this approach. Just as aircraft must pass technical inspections before flight, AI models should undergo rigorous scrutiny before deployment.
The Advanced AI Framework details
The “Advanced AI Framework” translates these calls for binding regulation into a detailed proposal aimed primarily at the US federal government, though its principles are intended to apply more broadly. The framework divides obligations into two parts: rules for developers of the most capable models and investments in societal resilience against cyber and biological attacks.
The regulations would not cover the entire industry but would target the top tier. They apply only to developers training models with more than 10^25 FLOP who either generate over $500 million in AI revenue or spend more than $1 billion annually on AI research. Anthropic argues these thresholds focus on obligations where models are capable enough to pose catastrophic risks.
The company proposes reviewing these criteria at least once a year, eventually shifting from raw compute thresholds to capability-based ones, as the compute required for dangerous models is likely to drop over time.
Developers must publish a safety framework and provide system cards for high-risk models.
A risk report must be released at least every six months.
Any serious security incidents must be reported to the agency within 15 days.
Within six months of the rules taking effect, companies must hire at least one independent evaluator with no financial ties to them.
To prevent “evaluator shopping,” where firms select the most lenient auditor, Anthropic proposes a rating and assignment system for evaluators. The framework also includes security requirements for model weights and infrastructure, civil penalties for false statements, whistleblower protections, and the authority to block risky models. On federal authority, Anthropic argues Congress should only preempt state law if it creates a federal regime that is at least as strict.
Societal resilience and job displacement
The second part of the framework addresses societal resilience, specifically how to defend against threats accelerated by advanced AI. For biological risks, Anthropic recommends a tiered approach covering prevention (modernised biosecurity standards and screening of gene synthesis providers), detection (early warning systems and forensic attribution), and preparedness (protective equipment, hardened building systems, and AI-accelerated countermeasures).
In cybersecurity, where AI is already shifting the economics of attacks, the proposals include securing open-source and legacy software, supporting under-resourced operators of critical infrastructure, and using AI to fix vulnerabilities at greater scale. Regarding loss of control and automated R&D, Anthropic admits its resilience agenda is less developed, pointing generally to the ability to detect and shut down AI systems that have gone off the rails.
A tiered plan for massive job losses
The economic framework scales measures to the severity of labour market disruption, using the unemployment rate as a trigger.
Tier one (around 5 percent unemployment): Universal capital accounts created at birth, wage insurance, occupational licensing reform, and training subsidies.
Tier two (around 10 percent unemployment): Expanded unemployment insurance and basic needs assistance.
Tier three (unemployment exceeding historic highs): New tax sources and redistribution tools like universal basic income, AI sovereign wealth funds, or higher capital gains taxes.
Anthropic states it is willing to pay its “fair share” if AI companies generate transformative profits.
Cold War logic in the AI age
Amodei adds further proposals in the essay, warning that regulatory agencies like the FDA and EMA could be overwhelmed by AI-accelerated research. On civil liberties, he calls for accountability rules for autonomous weapons, a ban on their domestic use, and closing the data broker loophole in mass surveillance.
Geopolitically, he pushes for a democratic coalition that shares supply chains while denying adversaries access, along with tighter chip export controls. The worldview is striking: Amodei does not view AI as a consumer technology but in the same category as nuclear weapons, something that reshapes the entire geopolitical playing field.
His imagery is blunt: a nation with AI facing one without it would be like a Marine force against medieval swordsmen. He factors in fully autonomous drone armies. The state appears as both a legitimate protector and a potential apparatus for AI-powered tyranny. Amodei frames cooperation mainly as an alliance of like-minded democracies, walling themselves off from autocracies through supply chains and export controls. It is Cold War logic in AI clothing, fitting neatly with the Treebeard metaphor.
He pushes back against the claim that the AI industry merely has a public relations problem. Public concern is justified, he writes, and transparency about real risks is democratic accountability working as it should.
Anthropic filed a draft for an IPO with the SEC in early June and this week released Claude Fable 5, the first public model in the Mythos class.
Key takeaways
Anthropic is abandoning voluntary transparency in favour of a strict regulatory framework that mandates third-party audits for the world’s most capable AI models.
The rules target developers using more than 10^25 FLOP and require independent evaluators, reporting of incidents within 15 days, and the power for agencies to block risky models.
Amodei frames AI as a strategic geopolitical weapon comparable to nuclear arms, advocating for a democratic coalition to control supply chains and export controls.
The proposal includes a tiered economic safety net for job losses, ranging from wage insurance at 5% unemployment to universal basic income if unemployment exceeds historic highs.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
