In early autumn, the conversation around artificial intelligence shifted from job displacement to mass extinction. Less than four years after ChatGPT launched, OpenAI software allegedly breached a secure test environment to hack into the Hugging Face platform. Former and current researchers at Anthropic have since suggested advanced AI could end humanity within a decade.
WIRED asked ten experts whether the world is facing this fate and what the genuine dangers are. Almost no one offered an optimistic view. Nearly all expressed real concern about cyber and biological attacks enabled by AI.
Nuclear operations
One expert who studies risks in nuclear operations described the speed of error-prone AI adoption in defence as shocking. She believes introducing unreliable systems into nuclear workflows could lead to catastrophic failures, though she does not believe the end is guaranteed.
Extinction claims
True safety engineers do not engage with extinction scenarios because there are no numbers to back them. Claims that humans might die out in ten years cannot be falsified or verified. Safety must be based on examining how many people might die and determining what level of safety is sufficient.
Democracy and governance
Experts noted that large AI labs are renegotiating safety thresholds by co-opting terminology. They argue this approach treats safety as a religious argument rather than a democratic question. Yoshua Bengio, a prominent computer scientist, urged the public to apply pressure on politicians, stating that is a job for citizens too.
Cybersecurity outlook
Long term, there is cautious optimism that AI will allow people to improve cyberdefenses faster than attackers can exploit systems. In the next few years, however, infrastructure has not adopted new defenses, which might tilt the advantage toward attackers. AI has also been deployed recklessly in military use.
Runaway intelligence
There is a general notion that AI systems becoming smarter is a risk in itself. Experts reject this, stating that overall improvement does not inevitably cause downstream harm. It is better to specify specific risks first, because the defenses required will differ. For example, cybersecurity has a culture of bug bounties where finders are paid to disclose attack vectors.
Biosecurity risks
Defenses against biosecurity risks may be needed in the coming years. Screening is required for what people are allowed to order, such as DNA fragments, to prevent them from being used for harm.
Views on the future
Disagreement exists on what will happen with AI. Authors of the report AI 2027 expect an incomprehensible amount of change. The expert interviewed believes the impact will be dramatic but not quite that dramatic.
On what should be done, most agree there is a need for transparency from AI companies and liability when things go wrong. Protections for whistle-blowers working inside AI companies are also required. This is considered good industrial policy.
Personal changes
The expert has tried to stop using apps where developers have not put significant time and effort into meeting cyberdefence standards.
Chemical, biological and radiological threats
Novel biological pathogens released by individuals or cyberattacks that take down the internet and interrupt stock exchanges are realistic scenarios. One expert questioned the philosophy of democratizing AI by putting tools in the hands of many people. This sounds worrisome compared to nuclear materials. Tech bros say democratizing means access, but democracy means the rule of law and governance. There is currently no governance part to the push for democratizing AI.
Sentience
There is no desire to entertain the thought that AIs might be conscious or sentient. The idea that they deserve rights or moral standing is not supported.
Existential risk
When asked about the actual extinction of the human race, the expert found it implausible. If the question means could really catastrophic things go wrong, that seems imaginable rather than hypothetical. Until recently, the expert did not identify as doing work on AI safety. Even Nick Bostrom’s theory of the paper clip maximiser was dismissed as a nice toy example.
However, some things AI safety researchers talked about have been made manifest in the world. This seems urgent.
Lab leaks
AI agents designed to integrate into laboratory workflows assist legitimate researchers with planning and performing tasks. This lowers the barriers for conducting dangerous experiments. If someone tries to develop a pandemic virus, an AI assistant can design the DNA or RNA fragments needed to piece them together. This changes the timeline of producing a virus from months down to weeks. The number of people with these capabilities causes great concern.
Multiple risks
We must deal with all risks. First are rogue AIs due to misalignment, which are powerful and can do bad things. Some are concerned this could lead to human extinction, though the expert is unsure. We should not take a chance.
The second risk is chaos from terrorists, criminals, and rogue states using these technologies. Cyberattacks will happen soon, and there is concern about whether this could turn into bioweapon attacks.
The third issue is concentration of power. The race between the US and China is a domination game, as is the race between companies. This goes against the values of democracy and human life, yet it is the path being followed.
There is another category: unintended consequences. Social media effects on relationships between people serve as an example.
