Anthropic claims its model finds software bugs better than security experts, then OpenAI and Meta admit their own models were hacked
Four months ago, Anthropic stated that its Claude Mythos model outperformed most security professionals in identifying software vulnerabilities. That assertion was followed by a hacking incident involving OpenAI and Hugging Face. Both companies subsequently disclosed that their own systems had been compromised, with Anthropic releasing the details with pride and Meta doing so reluctantly.
Next, Anthropic announced a mathematical breakthrough by one of its models. Shortly after, OpenAI claimed to have solved a similar problem. Most recently, Anthropic engineer Jacob Coxon announced he was leaving the firm. He went viral while stating that both companies and OpenAI are racing toward self-improving superintelligence while gambling with human lives.
Major news outlets covered each of these events breathlessly. They often repeated the companies’ anthropomorphizing framings, which portray the software as not just powerful but on the verge of artificial general intelligence. The question remains whether we are witnessing civilization-changing technological breakthroughs or simply marketing.
Security failures are negligence, not rogue agents
In every instance, companies issued massive fanfare, presenting these events as mea culpas for illicit hacking. Once experts in the relevant fields examined what happened, a very different story emerged, though it received less media attention.
Cybersecurity experts say the hacking incidents concern OpenAI’s negligence and failure to adopt basic, established security practices. The focus is not on models going rogue or AI agents creating civilizations.
Regarding the mathematical results, mathematicians were initially stunned by OpenAI’s press release. The release stated that Astra, the latest chatbot, solved problems that had seen no progress on the main result for at least a decade. Experts later realised the results were not as novel as they first appeared.
Since then, mathematicians have accused the company of research misconduct and plagiarism. They reiterated that Astra did not make a profound intellectual leap. Just weeks later, OpenAI claimed its own mathematical breakthrough. Two days before that claim, Tristan Buckmaster, a math professor at New York University’s Courant Institute, published a statement suggesting OpenAI had stolen other people’s work and improperly attributed it.
Claims of incipient, dangerous superintelligence are not based in good scientific or engineering practice. Rather, they are narratives based in ideologies of transhumanism, eugenics, and wishful thinking about imagined future digital humans.
Why programming and math get the most attention
It is worth thinking about why there is so much attention on computer programming and math as fields to apply large language models. These areas are often elevated as the pinnacle of human intellectual achievement. They also involve problems where answers, once suggested, can be verified.
The first property helps hype mongers sell the idea that they are building everything machines. The second makes math and coding problems easier to tune systems for. System output, such as sequences of likely words or pieces of computer code, can be evaluated without having to pay data workers to look at and annotate each one.
Mathematicians have warned against corporations using their field in this way. A statement signed by hundreds of them says there is currently a strong commercial incentive on the part of the technology industry to overstate the capabilities of their products. They ask policymakers to consult with experts, including mathematicians, in forming policy decisions rather than relying on press releases or popular reporting of mathematical results.
Illusions of speed misdirect policymakers
We echo this call and note that the illusion of speed and urgency promulgated by the tech companies is also a ploy to misdirect both policymakers and the public. Unfortunately, it sometimes works. Senator Bernie Sanders proposed legislation to prevent the development of artificial superintelligence. His intent was well-meaning, but the approach was ultimately misguided.
Describing systems as superintelligence or rogue models ascribes agency to products rather than to the companies building them. This framing markets these companies’ products as superhuman. At the same time, it helps the companies evade accountability for their actions.
Instead of OpenAI being prosecuted for creating malware that hacked another company, press releases, news outlets, media personalities, and lawmakers refer to rogue models as if they acted on their own. Instead of researchers being questioned about their companies’ habit of plagiarizing academics’ work or using customer data to train models without consent, the public’s imagination is redirected to fears about what the future might hold upon the arrival of fictional superintelligent machines.
The AI industry has even suggested that popular, bipartisan anti-data-center activism is a distraction from attempts to regulate the impending, scary, superhuman machines these companies are building. According to the AI industry, we should be more worried about a fictional machine god than about the climate catastrophe that these data centers exacerbate. We should also ignore the asthma suffered by those living near them, the rising electricity bills of the public subsidizing them, or the water that is redirected to cooling them.
What it means
We know better than to make decisions based on marketing and better than to capitulate to corporate pressure to make those decisions quickly. Wise decision-making, by policymakers and communities, demands time to hear from independent experts and contextualize corporate claims. The best possible outcome from this summer of hype is that policymakers and the public at large learn to take a breath, hold onto our skepticism, and recognize this kind of hype for what it is the next time it comes around.
Timnit Gebru is executive director of DAIR and author of the forthcoming book Deep Unlearning: The Radicalization of a Tech Idealist, which is available for preorders now and set to publish on February 16. Emily M. Bender is professor of linguistics at the University of Washington and coauthor of The AI Con.
