Dario Amodei, founder and CEO of Anthropic, clarified on Monday that his company does not support bans on open-weight AI models, despite industry whispers suggesting otherwise.
Amodei’s clarification
Amodei wrote that anyone reading his previous work would know he does not view such bans as useful. He added that Anthropic has never advocated for them.
The comment followed a post by Nvidia founder Jensen Huang on X. Huang shared an open letter on Friday urging policymakers against broad restrictions on open-weight models. The letter came from Nvidia and a list of other companies including Hugging Face, Meta, Microsoft, Mistral, and Nvidia. It warned against premature rules.
The letter did not mention China. Yet the industry debate focuses on allegations that Chinese labs steal intellectual property from American rivals. One method is distillation. This occurs when an AI bombards another model with prompts to learn its workings.
Distinguishing the threat
Amodei stated in his blog post that open-weight models fall into a different category than the threat China poses. He described open-weight models without dangerous capabilities as a public good. They cost nothing besides the compute needed to run them and provide value to businesses, developers, and researchers.
Amodei noted he has longstanding fears about AI. However, businesses using open-weight models, even from China, are not among his concerns.
The real worry
His fear is that authoritarian governments will build models more powerful than those in the U.S. to achieve permanent military superiority. He also worries about the use of AI to repress their own people. He said the Chinese Communist Party is not the only authoritarian government he is concerned about, but it is the most capable.
Amodei also fears AI will enable biological attacks, not just cybersecurity ones. He believes open-weight models are more dangerous in those scenarios because it is difficult to apply guardrails or monitor their usage. He noted, citing a UK AI Security Institute report, that once open-weights are released they cannot be withdrawn.
Open source defenders
This runs counter to what open source advocates argue. They claim that access to powerful open models not controlled by a single entity helps defenders protect themselves.
Proposed solutions
Amodei listed actions he believes would thwart China. These include restricting its access to powerful chips, which has been a longstanding U.S. policy. He also called for a formal crackdown on distillation. The U.S. has threatened sanctions against China if it determined there was IP theft involving U.S. models.
Amodei also said he supports growing efforts to create a model safety testing organisation. Some of these efforts are led by the U.S. He said this would be especially effective if the entire world, including China, agreed to submit to it.
He wrote that he thinks this idea is close to a consensus. He noted that the Trump administration has moved in this direction in recent months. He added that testing would need to be global to be effective, which means even the CCP would need to be on board. He suggested limited cooperation around preventing AI biological weapons may be possible because it is in China’s interest too.
What it means
Anthropic wants to stop the spread of dangerous AI without stopping the open sharing of safe models. The company sees a clear line between tools that help developers and those that could be used for military or biological harm. This approach separates technical openness from national security risks.
