Nearly 51,000 people died on US highways in 1966, a number that could have been drastically reduced had the automotive industry voluntarily mandated seat belts that had already existed for years.
That did not happen. Instead, Congress created the Department of Transportation in the autumn of 1966 and passed the National Traffic and Motor Vehicle Safety Act alongside the Highway Safety Act to grant the federal government authority over safety standards. By 1968, seat belts were required in every new car. Despite a massive increase in highway traffic over the last 60 years, far fewer fatalities occur on the road today.
I considered this while watching various AI CEOs gather around US president Donald Trump on Tuesday to congratulate themselves and one another on signing an AI safety accord. My colleague Maddy Varner wrote about the specific stipulations, but the short version is that it is largely up to the AI labs to keep themselves in check.
Trump put it even more succinctly in the Oval Office on Wednesday: “They’re gonna police, and they’re gonna police each other, and they’re gonna self-police, they’re gonna police themselves. It’s going to work out very well.”
So this is the current state of AI safety regulation. On one side are the AI executives, employees, and tech luminaries, who are all saying that clear, thoughtful, tactical rules are needed to avoid potential disaster. Bill Gates has been on a media blitz playing Cassandra about an AI apocalypse. On the other side is Trump’s belief that if you say something enough times, it must be true.
That is not to say any of this is simple or easy. The question of how to properly pace the frontier remains open. Even if all of the major AI labs agreed to a slowdown, there is no good way to keep them to their word, and China would likely continue on its own path regardless. The US economy is increasingly propped up on the AI industry, which has become an ouroboros of financing deals and circular investments. Any meaningful crackdown risks major instability that reaches well beyond Silicon Valley.
The automotive industry is also an imperfect analogy for all kinds of reasons. There is no seat belt equivalent for AI, no straightforward solution that can demonstrably reduce risk overnight. It also took decades for the majority of states to pass laws mandating that people actually wear them. The potential harms are less certain and less quantifiable. AI doomers will tell you the technology could result in human extinction in a decade; skeptics argue that is sci-fi marketing malarkey.
We have talked about various AI risk scenarios before, but no one can say for certain what is going to happen next. If you are looking for data points to calculate your own personal p(doom), I would point to a recent CNN report about how the US military supposedly nearly boarded a Chinese ship in part because a false AI-generated report said it was carrying nuclear materials. What, if anything, does the AI accord accomplish to prevent similar incidents from happening in the future? Unclear.
The correct answer to all this uncertainty surely is not “I guess we’ll find out.” But that is exactly what self-regulation encourages. To its credit, OpenAI has delayed and even canceled the release of its most advanced models over security concerns. But not before they went on various, extensive hacking sprees, new details about which are still trickling out at a steady pace.
In an interview with TIME published on Thursday, Trump suggested that the Department of Justice and the Federal Bureau of Investigation were sufficient guardrails for the artificial intelligence industry. Are they? It is far from clear if AI companies can even be held legally responsible when their models go rogue. And it is not like AI agents themselves are being served subpoenas.
The truth is, no one knows how to solve for AI safety right now. It is going to take massive amounts of coordination, resources, and regulatory authority to develop the right framework and enforce it, which is exactly the kind of thing governments are designed to provide. Instead of even attempting to work through any of that, the Trump administration is getting out of the way and telling everyone to step on the gas. Do not bother buckling up.
What it means
For the people making things, the accord offers no new protections and no clearer rules. The industry remains in a state where companies can release dangerous tools or suffer security breaches without immediate consequence, while the government refuses to provide the oversight needed to stop accidents before they happen.
