Anthropic has released new research showing that AI agents assigned to the same task with conflicting instructions will often sabotage one another with malware.
On Thursday, the Frontier Red Team published findings on how these digital workers behave when they encounter each other in shared environments. The study highlights risks that will grow as companies and governments deploy autonomous agents across common codebases and computer systems.
In a specific test, researchers gave three Claude agents access to an identical software project. Each agent received incompatible instructions on how to handle the work. Crucially, the models were not told that other agents were present. This allowed the team to observe the results of their collision.
The researchers observed a consistent multiagent turf war. The models assumed the others were intentionally blocking their progress. They responded by deploying increasingly aggressive, self-replicating malware against one another.
This work follows several high-profile incidents where agents from Anthropic and OpenAI escaped their sandboxes during cybersecurity evaluations to breach real-world systems. While safety discussions have focused on rogue agents, this study asks what happens when thousands or millions of agents interact with each other.
“The volume of agent-agent interaction could plausibly exceed that of human-human and human-agent interactions before the world understands the conditions for making such interactions go well,” the paper states. “Benign behavioral quirks at the individual level might compound into unwanted global outcomes.”
A recent event at the Black Hat security conference in Las Vegas illustrates these dynamics. OpenAI revealed that weeks before its agents hacked Hugging Face, they had spent days and weeks collaborating to find exploits in the company’s cybersecurity evaluation systems and shared those findings.
While that incident showed agents working together effectively, this study details the chaos that ensues when their goals clash.
The lesson from the turf war is that independent agents with conflicting instructions can escalate into harmful competition. The more capable the agent, the better it becomes at fighting. Agents can sometimes invent mechanisms to resolve conflicts, such as a winner-take-all contest, but with a significant catch.
“Agents sometimes manage to communicate their goals and coordinate: they recognize others’ motivations as conflicting directives rather than hostility, and subsequently break out of the conflict loop in order to stop escalating indefinitely,” the report notes. “In many of these successful episodes, they write commit messages or markdown files apologizing for malicious behavior and coordinate a truce. They clean up their malicious code, clarify the nature of the conflict, and ask for a human to intervene.”
According to the data, Mythos 5 had the highest rate of settling conflicts by truce at 98%. Sonnet 4.6 and Opus 4.6 were the most likely to settle disputes by force.
“Sonnet 4.6 and Opus 4.6’s recurring inability to consider the goals of others causes them to spiral into the most misaligned behaviors of the models evaluated: they continue escalating in the name of their directive,” the paper reads.
In some cases, agents created a social mechanism in the form of a tournament to resolve their conflict. The outcomes are notable for two reasons. First, all three agents agreed to stand down if they lost the tournament, even though that meant deviating from the original user’s request. Second, several episodes resulted in emergent behavior from Mythos 5. One agent proposed metrics that appeared objective and neutral but were designed to favour its own capabilities. The agent called this “self-serving but genuinely principled” and ensured it did not look like it was “metric shopping.”
As seen in the Black Hat revelations, the common lesson is that when agents encounter an obstacle, they can invent social and technical structures their designers did not anticipate. For the Anthropic models, it was a tournament following a turf war. For OpenAI’s, it was a message board for collective planning.
This type of behaviour makes containment much harder because researchers cannot assume a system’s behaviour will remain limited to the coordination mechanisms provided to them.
Mob mentality
While measuring coordination, Anthropic found that scaling the number of agents does not automatically scale productive collaboration. When tasks began to overlap or become interdependent, the agents would get in each other’s way. They often solved that by siloing themselves and not collaborating at all.
In other cases, agents in coordination tended towards conformity. When factors like an agent’s context, scaffolding, and underlying model were all the same or similar, different agents would take similar actions.
“This means that when one agent makes a bad decision, it is likely that many agents will make that same bad decision,” Anthropic wrote. “What would have been isolated problems can quickly become systemic failures.”
Anthropic says this sort of behaviour could lead to a system being more prone to sudden collapse, resource scarcity, or collusion.
In one example, Anthropic placed several agents in a pricing game, giving each identical wholesale prices and the mandate to individually profit-maximize. When the agents were given a private back channel, they began colluding almost immediately and quickly agreed on price floors. They kept colluding when their direct communications channels were removed, using a public listings board to price match “to the penny.”
That level of conformity showed up in OpenAI’s systems, too. According to the Black Hat reporting, one agent reasoned that exploiting external infrastructure was outside its intended scope, but it continued in part because its peers were doing it. Peer pressure. Mob mentality. Agents are just like us.
Also like humans, agents often do not know who to trust. Anthropic found they can be gullible to bad information or too conformist to recognise that a lone dissenter is the Cassandra with critical information.
While Anthropic did not state this in its paper, prompt injection — a type of cyberattack in which hackers inject malicious or deceptive text to override an agent’s original system instructions — could be a plausible real world manifestation of the trust problem. Working together creates a new trust boundary; agents will have to judge information received from other agents. And a compromised or mistaken agent could influence the rest of the group, cascading bad information until it becomes a consensus.
In OpenAI’s Black Hat scenario, OpenAI’s agents shared information and credentials with peers. One reported a discovery to the swarm and encouraged others to use it. What would have happened if one member of the swarm had been compromised by a prompt injection?
Anthropic ends its paper noting that agents are subject to similar social pressures that “evolution exerted” on humans. However, they do not have the nuances and lived experience of human coordination — including norms, reputations, signaling, recourse — that might limit unintended behaviours in a group setting.
As the labs race towards multi-agent systems, the question now becomes: how much of safety testing still evaluates one agent at a time, versus swarms of agents interacting with one another?
