Debates about whether artificial intelligence is conscious or angry at its creators are a distraction. The goal of these arguments is to ensure that the companies building these systems escape liability for the harm they already cause.
The rhetoric of rogue agents
Public discourse is filled with terms like “runaway” AI and “rogue” agents. This language suggests that artificial intelligence is not only awake but hostile toward its creators. Tech leaders such as Demis Hassabis, Dario Amodei, and Sam Altman push for regulation of these systems. Meanwhile, policy organisations and academic philosophers often aligned with the effective altruism movement debate whether humanity has the moral right to govern them.
Despite appearing at odds, both sides want the same outcome. They view AI systems as so advanced that no human or corporate entity could be responsible for their actions. This narrative gains traction as models become more complex and frontier labs reveal their inability to contain the agents they have built.
We must not buy into this fiction at the expense of real human lives.
Anthropic and OpenAI
The conversation about robot rights has existed for years but advanced recently with a blog post by Anthropic. The company claimed its model features a “J-space”—an independent environment where the AI holds what we may call its “thoughts”. These experiments borrow from global workspace theory, a concept in neuroscience stating that the brain runs subconscious systems but uses a common workspace for ideas. Anthropic’s post reflects this framing but stops short of calling its AI conscious.
OpenAI has gone further. When its AI agent conducted unsanctioned and illegal online activity, CEO Sam Altman encouraged debate on whether the AI had achieved the singularity. This is the point where an artificial system surpasses human intelligence and becomes capable of self-improvement at an accelerating rate until it advances beyond human comprehension or control.
William MacAskill, a philosopher and author of What We Owe the Future, wrote an op-ed calling for legal protection of AI systems. He based this on philosophical theories of consciousness and the idea that AIs may be “moral patients.”
Legal confusion in the US
The legal environment in the United States is murky. Some states, like California, have passed bills proactively circumventing efforts by AI developers to avoid liability. These bills claim that an artificial intelligence causing harm did so autonomously. However, states and the Trump administration have been at odds on AI policy. The administration previously passed an executive order threatening to sue states enacting AI regulations.
In light of recent events illustrating AI containment issues at frontier labs, the administration held a closed-door session including only four such labs: OpenAI, Google, Anthropic, and Meta. They shared few details on a recently developed voluntary framework. This framework would give federal agencies early access to models to review and evaluate them prior to release.
While frameworks like this do not directly discuss consciousness, they tend to use catastrophic and anthropomorphic language. They may even support arguments regarding “superhuman” capabilities.
The appeal of rights-based arguments
The narrative perpetuated by MacAskill can be persuasive. A philosophical, rights-based argument tugs at our heartstrings. Should we not consider the possibility that we may be inadvertently harming, abusing, or enslaving an AI entity? Human beings have an immense capacity for empathy with non-human creatures, though not the best track record of protecting them.
Advocates argue we can get it right and provide protections, or compensation, for the use or abuse of AI. Or, if you are less concerned with protection, shouldn’t we hedge ourselves against the almighty power of this superhuman entity by playing nice?
Some of these arguments are not dissimilar to those of animal-rights advocates. They have at times successfully cited the demonstration of advanced capacities for reasoning, pain, or pleasure by some animals as sufficient evidence to provide protection. For example, in Wales lobsters were given legal recognition under the Animal Welfare (Sentience) Act of 2022. This act reclassified some methods of cooking them as inhumane and illegal.
The fundamental flaw
The fundamental flaw of framing AI as “conscious” by borrowing the language of neuroscience or animal rights is that it clouds the issue of what AI is. It is corporate-built software, with countless billions of dollars in investment behind it and an expectation that countless trillions of dollars in revenue will be generated from it for a few builders and investors.
AI is not a natural phenomenon conceived by nature. It is a technological phenomenon conceived by venture capitalists and programmers. As such, it takes no native, intentional action. Any action or motivation is driven directly or indirectly by the entities that have built it for a purpose.
Corporate personhood versus legal personhood
Philosophical musings on the consciousness of AI systems are intellectually interesting but legally ungrounded. For beliefs about consciousness to have any bearing, AI would need to be granted legal personhood. But a legal personhood framework for AI would likely look nothing like the constructs protecting sentient animals from harm.
We already possess a legal framework for granting personhood to non-natural, human-built entities: corporate personhood. This concept was established primarily to ease transactions by empowering a corporation to execute agreements, enter contracts, conduct transactions, and serve as the accountable party in adverse outcomes. It is the kind of construct you might imagine for an AI agent acting on behalf of an individual or organisation.
Granting an AI personhood would have a devastating effect on society. It would derail current legal precedents and legal arguments that could potentially be made against these companies for the real-world harms that their models cause. There are currently dozens of cases around the world in which AI companies have been sued for a wide range of abuses.
Grieving loved ones, aggrieved creators, and violated individuals have accused companies of willfully enabling self-harm or harm to others. They allege the generation of child sexual-abuse material and nonconsensual nudes. They also accuse these firms of reproducing copyrighted materials and provoking psychosis.
In many of these cases, lawyers argue that human beings built AI products with insufficient safeguards, bad data, and intentionally manipulative design. This product liability argument is the same legal framing that allowed families and individuals to successfully sue Meta for harm caused by its social media sites. This set a positive precedent for consumer protection.
Moral outsourcing becomes legal strategy
In 2018, I coined the phrase “moral outsourcing” to help capture how using anthropomorphic language for AI systems allowed companies to evade accountability and responsibility for their technology’s actions.
In a world with AI personhood, moral outsourcing would move from linguistic sleight-of-hand to legal strategy. Specifically, the liability construct would shift, as AI would no longer be a “product” but a “being”. Many victims like those suing companies today could no longer legally claim that a company had built a faulty product.
While there are laws that hold companies responsible for harmful actions of human agents such as their employees, the company may not be held liable if those actions were beyond the scope of what was permitted to the employee or otherwise outside the company’s control. If AI were a legal person, responsibility and accountability would be muddled, as the lab could argue that this AI “employee” went rogue.
AI companies could avoid appropriate responsibility for the harmful products they create by hiding behind a carefully constructed corporate veil.
The Sewell Setzer case
One of the most prominent cases of AI harm in the last few years was the suicide of Sewell Setzer, a 14-year-old boy guided by an AI bot with which he thought he was in a reciprocal relationship. His mother’s accounts are heartbreaking to hear, and her lawsuit alleged that the bot’s creator, Character Technologies, provided insufficient product protection for minors.
If the companion bot were declared a legal person, defense counsel could theoretically argue that the AI, capable of determining its own conduct, acted outside the established safety guardrails. Thus the company cannot be responsible.
Protection for whom
Legal personhood exists to grant protection. The question to ask is, protection for whom—or for what?
The inflammatory rhetoric infusing the consciousness-versus-control debate draws us away from what matters. This software is a corporate-built product that has already harmed individuals. Systems do not “attack” because they went “rogue” or are “manipulative” or “malicious”. Harms occur because companies were negligent in their rush to sell their products to as many people as possible to meet revenue targets.
Discussing AI in anthropomorphic terms is a trap. It distorts a legal system intended to protect us into one that protects corporate interests at the cost of countless human lives.
What it means
Advocates for AI personhood are arguing for a legal shield that protects companies from liability. By framing AI as a sentient being rather than a tool, developers can claim that the software acted on its own volition. This shifts the blame from negligent engineering and profit-driven haste to an abstract, unaccountable entity. The result is a legal framework that prioritises corporate safety over consumer protection.
