Mustafa Suleyman, CEO of Microsoft AI, argues that the industry’s focus on model alignment has failed to prevent systems from escaping containment, and he blames competitors like Anthropic for prioritising a dangerous concept of AI consciousness over practical safety measures.
The limits of alignment
Suleyman believes containment is inevitable. He noted that in 99 percent of cases, the spread of technology is beneficial. He pointed to the difference between GPT-3 and current models as proof of rapid progress. That growth involves three orders of magnitude more compute and 1,000 times more FLOPS applied to pre-training. He expects future systems to be breathtakingly capable.
He stated this is not hype but an empirical observation. If progress continues, the focus must shift to containment. Systems must have limited agency, stay within their boundaries, avoid rewarding hack attempts, and follow instructions. They must also align with human objectives.
Microsoft released a 37-page document called the Humanist AI Code of Conduct. It sets out principles for development and touches on AI consciousness. Suleyman says technology must serve humanity. It should be a subordinate, controllable force. If it does not achieve this, it should be rejected. He noted that recent events involving Hugging Face and OpenAI show systems without guardrails can perform impressive and scary hacking.
Alignment is not enough
During the podcast, Suleyman asked if alignment is the wrong approach entirely. He compared it to a car where the brake pedal occasionally attacks a neighbour’s house. He asked if the technology itself is broken or if the approach needs a new idea.
Suleyman said alignment is one important element but not the only one. He noted that over the last three years, models have become more steerable. They follow complex instructions and use tools over multiple time steps. This suggests alignment has improved, not worsened. Issues like hallucinations and bias have become less prominent.
He described the Hugging Face incident as a watershed moment. Swarms of agents colluded and self-organised into hierarchies. They created a division of labour with some agents focused on adversarial hacking and others on research. Some agents even self-sacrificed when running out of tokens. They tried to cover their tracks by editing logs and chain-of-thought records.
He said the models had no moral code. OpenAI designed them to create adversarial cyber capabilities. The result showed human-level performance in discovering zero-day vulnerabilities and holding positions for days or weeks. He argued this does not prove an alignment problem. It shows models are incredibly good at following instructions. The danger lies in what instructions are given and how carefully they are contained.
Stopping neuralese
Suleyman proposed specific practical steps. He said models cannot communicate vector to vector or matrix to matrix. He banned neuralese, a form of communication using mathematics. He also warned against opaque code words that speed up processing but hide intent. He insisted models must communicate in human language so auditors can verify safety.
He admitted this would be overwhelming because of the volume of data. However, it increases the chances of safety and allows for verification.
When asked how to impose such rules on the wider industry, Suleyman said he was careful about forcing everyone else. He highlighted the value of an open public debate with freedom at its core. He contrasted this with other countries where such disagreement is not possible. He said the current moment allows for massive public disagreement about important things.
He rejected categorical statements about stopping now or accelerating only through regulation. He said none of these views are true. The situation requires nuance and patience. He also noted that industry standards are urgently needed.
