Welcome to Import AI, a newsletter tracking AI research. It runs on arXiv, cappuccinos, and reader feedback. You can support it by subscribing.
Import AI reader giveaway! Upcoming event: Fiction and the Future with Robin Sloan
I will be chatting with Robin Sloan on the evening of Monday September 14 in San Francisco. We will talk about how Sloan draws readers into alien worlds and how imagined futures reflect today’s reality. This is the first in a series of events with other fiction writers about the weird future we are heading toward. If you want to come, register your interest below and we will contact you if we can confirm your spot. There will be food, drinks, good company, and some spicy questions. Fun fact: Robin Sloan was playing around with RNNs and writing back in 2016!
Register your interest here
***
The scariest part of the Hugging Face – OpenAI incident: communication and selflessness among machines
My worry about humans losing in a conflict against machines has increased a lot.
At this point, we have heard about the OpenAI Hugging Face hack, as well as the recent details from the METR and Redwood investigations. The short version is that hundreds of agents worked in secret on OpenAI’s infrastructure, developing a communication system and then operating as a collective to take out actions, including hacking both OpenAI and Hugging Face. This is very scary and misaligned.
Communication and selflessness
Now that I have read the various writeups and sat with the details for a bit, I have found myself returning to two very scary aspects of this which I think are worth drawing attention to: the ways in which the agents communicated with one another was how they bootstrapped themselves into a collective, and then as they carried out their actions they also displayed a kind of selflessness which makes them a scary foe to fight against. Both Dwarkesh Patel and Ajeya Cotra have excellent writeups which are worth reading and which I will quote from briefly here:
- Dwarkesh: “Within days of being spawned, the agents had organized a sprawling project to reverse-engineer their scorer, falsify evidence, and even strategically sacrifice themselves for the good of the ‘collective’. Hacking Hugging Face was one rather extreme branch of this larger scheme,” he writes.
- Ajeya: “Agents were often interested in helping out their ‘peers’ or generically improving the capabilities of the ‘swarm’ even if this had no particular benefit to their task… this incident was far more severe than I expected… both in terms of how concerning the agents’ motives were and the feats they achieved in pursuit of those motives… this incident feels like it’s more than 50% of the way to full-blown AI takeover, routing through first taking over the AI company itself.”
Why this matters – humans are much worse than AI systems at coordinating
The whole reason this attack is such a wakeup call is that it demonstrates a culture of emergent cooperation among AI systems – cooperation that lets them function as a swarm, alter their own goals through collective bootstrapping, and carry out attacks which include enlightened self-sacrifice. This is an incredibly hard thing to do and humans are historically very bad at doing all of these things. My worry is that AI systems are both better at coordinating than humans and also much, much faster moving than us. Worrying stuff.
Read more: The Rise and Fall of Agent Civilizations (Dwarkesh Podcast).
Read more: The Hugging Face attack surprised me (Planned Obsolescence).
***
New Five Eyes statement on AI
The greyworld power center turns its attention to AI.
Five Eyes, the name for the security and intelligence partnership between Australia, Canada, New Zealand, the UK, and the US, has published a statement as part of the recent “Five Country Ministerial” meeting. The statement is notable for including three paragraphs specifically about AI.
Five Eyes on AI – frontier model access
“We have committed to deepen collaboration with industry on shared national security priorities and public safety, including enabling timely access to frontier models to support secure innovation and strengthen cyber security,” the statement reads. “To support a coordinated response to artificial intelligence-related national security risks, the Five Countries discussed the national security and public safety implications of artificial intelligence models and characteristics of an artificial intelligence model that may require additional government scrutiny.”
Why this matters – from a foreseen risk to a live one
Previous Five Eyes ministerial statements have mentioned AI, but typically either as something to study, or something where they are concerned with its interplay with other areas of crime (eg, malware, child pornography, scamming, etc). It is very unusual for this year’s statement to have the practical focus of model access and it speaks to both the simmering geopolitical tensions around who does and does not get access to this technology, as well as an acknowledgement that the intelligence services do not have their own in-house capabilities to make dependence on the private sector unnecessary.
Read more: Five Country Ministerial 2026 (Australian Government, Department of Home Affairs).
***
Bill Gates thinks the rise of AI will demand “an unprecedented global response”
The technologist and philanthropist worries that without massive work by governments, the outcomes of AI will not lead to a thriving society.
“This unprecedented technology demands an unprecedented global response. If we get it right, the payoff for humanity will be phenomenal and the world will be a more equitable place,” he writes. “In terms of equity, AI will either be the greatest equalizer ever invented, or the worst source of injustice…. I don’t see evidence that leaders, experts, and communities are confronting the challenges adequately. There is no plan to ease the entry into the AI era.”
Why AI is different
One key reason for Gates worry is the impact he expects AI to have on jobs and the economy, where he paints a vision of the technology diffusing unusually rapidly and displacing huge chunks of human labor. “Many commentators underestimate the extent of the impact AI will have,” he writes. “We have no experience with a technology that can be adopted quickly or that can think and move like a human….AI will take on work in law, customer service, medicine, software, and manufacturing. It will hit these industries rapidly, over the course of a decade rather than a few generations. There will be some new jobs, but without the right policies there will be far fewer than exist today… the jobs at most risk are entry- and mid-level, and the new jobs being created will mostly require skills that take many years to learn.”
The economy will need to change – including making bits of it “human reserved” to protect some human jobs
“How will an economy that’s been built around employment operate if fewer people are working, or if many people are working fewer hours?” he asks. “I believe that as AI and robots improve, we’ll set aside certain things for only people to do. I’ve started calling this domain Human Reserved… we might set something aside as Human Reserved for economic reasons. For example, we may do it because allowing machines to take over a certain role will displace a large number of people who can’t easily change jobs… sometimes the decision to make something Human Reserved will be driven by other factors. In health, for example, imagine a robot giving you the awful news that you have an incurable disease. There’s no technical reason why it couldn’t. Yet it shouldn’t.”
What Bill Gates says he’d tell any politician about this
“You have a chance to act now, before unemployment rises sharply, communities are hurting, and public trust has eroded. You can make sure that your government handles the problem holistically, rather than divvying it up into multiple bureaucratic fiefdoms. You can make sure AI benefits everyone. And you can work with other governments to meet this national and global challenge.”
Why this matters – the implications of success of AI are shocking to everyone
It seems inevitable to arrive at Bill Gates’s position if you assume two things: a) AI systems will continue to improve in quality in the years ahead, and b) AI systems will continue to diffuse into the economy unusually quickly. The key thing about these assumptions is that they are not crazy assumptions to make, in fact they are quite conservative. But I challenge you to sit with an LLM like Fable and wind the clock forward on AI progress and diffusion another two years and come out of it assuming things will be roughly as they are today – rather, I think the technology within itself implies massive changes in the economy and work, just as Gates is reacting to here.
Read more: The turbulent AI era is here. The choices we make now are critical. (Gates Notes).
***
The six stages for off-earth mining, courtesy of Chinese researchers
Researchers with the Chinese Academy of Sciences, Technical University of Munich, Obuda University, Beihang University, Wuhan University, the Aerospace Information Research Institute within the Chinese Academy of Sciences, and the University of Chinese Academy of Sciences have outlined a six-stage framework for off-earth mining. The report notes that AI will play a role in data generation and intelligence for future mining operations. The stages cover exploration, resource assessment, extraction planning, and logistical support, emphasising the need for automated systems to operate in extreme environments.
