More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and Jakub Pachocki, have published a paper warning that automating artificial intelligence research could trigger an intelligence explosion. The authors argue that current systems already generate most of the code used to build them, meaning the entire research pipeline could become fully automated within years. This shift would accelerate progress from years to months, creating a scenario where society struggles to maintain control or understand the technology. The researchers fear that power balances between nations, corporations, and governments would erode as superhuman capabilities slip beyond human oversight. They specifically call for policymakers to gain significantly more visibility into how these automated systems are developed and deployed.
The warning adds to a growing chorus of concern from the scientific community regarding existential risks. Recent calls from mathematicians and internal fears from lab employees highlight that alignment problems remain unsolved, preventing responsible scaling at maximum speed. Some staff at Anthropic are reportedly looking for safe havens in case the technology goes wrong. The core issue is the speed of change outpacing the ability to regulate it.
* Progress timelines could compress from years to months
* Control over superhuman systems may slip away
* Policymakers lack visibility into automated pipelines
