For makers and artists: why AI’s creative potential demands new biosecurity guardrails
While the tech world often focuses on generative art and audio synthesis, a quieter but far more dangerous frontier is opening up. Leaders in artificial intelligence are now sounding the alarm that their tools could be weaponised to design lethal biological agents, a capability that renders traditional safety protocols obsolete. This is not science fiction; it is a tangible risk that requires immediate legislative action.
A call for mandatory screening
The chief executives of several major AI firms have penned a public letter to Congress, urging the adoption of stricter laws to prevent malicious actors from leveraging their technology to create biological weapons. Among the signatories are Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic, and Mustafa Suleyman of Microsoft.
The letter calls for legislation that would compel companies selling synthetic DNA and RNA to rigorously screen both customers and orders. The goal is to stop the misuse of genetic material before it leaves the lab.
The erosion of knowledge barriers
Organised by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation, the correspondence highlights a critical shift in the landscape. As AI development accelerates, the historical knowledge barriers that have protected society from biothreats are rapidly disappearing.
The first successful synthesis of DNA was achieved by scientist Arthur Kornberg in the 1950s. Today, that process is fully automated. Dozens of global firms use commercial synthesisers to “print” and sell custom genetic sequences for legitimate purposes like diagnostics and drug development.
While many providers restrict sales to qualified researchers, biotech firms, and educational institutions, the vetting process is inconsistent. Not all companies thoroughly check the identities of their buyers or the specific sequences they request.
History of alarm and the smallpox threat
Concerns were first raised in 2017 when Canadian researchers used roughly $100,000 worth of mail-order DNA to reconstruct the extinct horsepox virus. Critics immediately warned that the same methodology could be applied to smallpox, a deadly and closely related pathogen. Since then, the cost of gene synthesis has dropped significantly, making such experiments far more accessible.
When combined with advances in AI, it is now feasible to design novel toxins and pathogens using large language models. Although some biological training would likely still be required to construct a functional virus from scratch, the barrier to entry is lower than ever. Bioterror attacks remain rare but carry the potential for mass casualties, public panic, and severe economic disruption. A major fear is that an AI-designed pathogen could intentionally or unintentionally spark a global pandemic.
The failure of current screening
“AI tools enable a user to very quickly identify where to turn to order sequences that will not be subject to screening,” says David Relman, a microbiologist and biosecurity expert at Stanford University who signed the letter. He adds that with appropriate prompting, these models can also advise users on how to alter their orders to bypass detection systems entirely.
Other signatories include scientists, national security experts, and executives from gene synthesis firms Twist Bioscience and Ansa Biotechnologies. These organisations are members of the International Gene Synthesis Consortium, established in 2009 to promote voluntary screening practices. Many companies already employ software to flag “sequences of concern” that could increase an organism’s toxicity or disease-causing ability.
“If you have technology that is capable of synthesizing DNA, then you should ensure that it’s used responsibly, and part of that is making sure that you understand what you’re making and who you’re making it for,” says James Diggans, vice president of policy and biosecurity at Twist Bioscience. The company has long advocated for formal rules in this area.
Legal frameworks and their limitations
Federal guidelines introduced under the Biden administration mandated that scientists and companies receiving federal funding must order synthetic gene sequences from providers that screen purchases. A bipartisan bill introduced earlier this year in the Senate aims to require all US-based gene synthesis providers to screen orders and customers for bad actors and dangerous pathogens.
However, screening tools are not infallible. Last year, Microsoft researchers published a study demonstrating that AI protein design tools could generate potentially dangerous gene sequences that slipped past corporate screening software. The models suggested new protein structures with similarities to known dangerous pathogens.
The role of AI labs
Geoff Ralston, former president of Y Combinator and a partner at the Safe AI Fund, believes that AI labs utilising biology models must take responsibility for screening their own users.
“It should be very difficult, if not impossible, to ask a model to help you do something imminently dangerous,” says Ralston, who also signed the letter.
Relman agrees that regulation of screening procedures is only part of the solution. “Given that the screening may fail in some cases, we must then have other points of control,” he says. “That’s where the AI companies are going to have to step up.”
Key takeaways
- AI leaders are urging Congress to mandate screening for synthetic DNA and RNA orders to prevent the creation of biological weapons.
- Historical knowledge barriers against biothreats are eroding as AI accelerates the ability to design pathogens, a risk highlighted by the 2017 horsepox reconstruction.
- Current voluntary screening practices and federal guidelines are insufficient; AI labs must implement their own robust user safety measures.
- Regulations must evolve to address the reality that AI models can generate dangerous sequences that evade existing detection software.
