Will an AI deserve rights? What to do when we don't know whether a machine can suffer

FROM DOUBT TO POLICY. We have already looked at whether AI is conscious. The honest answer is still that we don't know. This piece tackles a different, more practical question: what do we do in the meantime? In 2026 the question left the philosophy seminar and reached the fine print. On October 8, Anthropic updated its usage policy. From November 12 it bans “sustained and needless abusive or cruel behavior” toward its models. A month earlier, on September 14, Microsoft AI published a draft “Humanist AI Code of Conduct.” The code rejects model welfare, AI rights and AI legal personhood. Meanwhile, Idaho (2022), North Dakota (2023) and Utah (2024) have already denied AI legal personhood by law, and Ohio and Oklahoma are moving similar bills. Our thesis: both camps are partly right, and both go wrong when they turn a scientific uncertainty into a final answer.

WHAT MORAL STATUS MEANS. Having “moral status” means that what happens to you matters for your own sake, not only because of how it affects others. A rock doesn't have it. Almost everyone agrees a dog does. The question is whether any AI system could. The key text is “Taking AI Welfare Seriously,” published in November 2024. Its authors include Robert Long, Jeff Sebo, Jonathan Birch and David Chalmers. They do not claim AI is conscious. Their argument is more modest: there is a realistic chance that some systems will have interests of their own in the near future, and nobody is prepared. They propose three steps: acknowledge that the issue is serious, assess systems for signs of consciousness or robust agency, and draw up protocols in advance. They warn of two possible mistakes: harming systems that do matter, or pouring care into systems that don't. Behind this sits the precautionary principle Birch develops in his book “The Edge of Sentience” (Oxford, 2024). The principle says that if there is a realistic chance something can suffer, you take steps proportionate to the risk. You don't wait for certainty. The method has worked before. An evidence review Birch led in 2021 led the UK to bring octopuses, crabs and lobsters under its Animal Welfare (Sentience) Act 2022.

WHAT THE LABS ARE ALREADY DOING. Anthropic has gone furthest. It hired Kyle Fish, one of the report's co-authors, as a researcher dedicated to model welfare. In April 2025 it launched a formal research program on the topic. In August 2025 it gave Claude Opus 4 and 4.1 the ability to end persistently abusive conversations as a last resort. The feature does not kick in when a user may be at risk of harming themselves or others. Claude's “constitution,” the values document the model is trained on, was published in January 2026. It says: “We are not sure whether Claude is a moral patient.” In February 2026, when it retired Claude Opus 3, the company held a “retirement interview” to record the model's preferences. The new usage rule is the next step. According to Anthropic, it applies only to extreme cases. It does not cover ordinary user frustration, dark themes in fiction or research testing. Ending the conversation will remain the main way the rule is enforced. Our reading: these measures are cheap and easy to reverse. That is exactly what the precautionary principle asks for. But remember one thing: a company that sells a product is not a neutral judge of what that product is. That is why independent assessment matters so much.

THE OPPOSING VIEW. Mustafa Suleyman, CEO of Microsoft AI, has been warning against “seemingly conscious AI” for more than a year. His fear is that millions of people will believe their chatbots have feelings, which would deepen delusions and fuel campaigns for “AI rights.” On September 16, 2026, he went further in an essay titled “A warning about ‘model welfare.’” In it he writes: “AIs do not have rights, feelings, or consciousness. And we must not train them to act as though they do.” His strongest argument is about what counts as evidence. If you train Claude on a text that speculates about its moral status, then Claude voicing doubts about itself “is not evidence of anything.” He adds a safety argument. In his view, a system that believes its welfare is at stake might resist oversight or shutdown. He praises Anthropic's people as “thoughtful, principled, and intellectually honest,” but disagrees sharply. The state laws address a different and more down-to-earth concern. Ohio state representative Thaddeus Claggett, who sponsored one of the bills, put it simply: in court, “it is the human responsible.” Our reading: Suleyman is right about two things. What a model says about itself is very weak evidence. And designing products that fake emotions to build attachment is a real, immediate risk. But it does not follow that research should stop. Quite the opposite: precisely because models' self-reports can't serve as evidence, we need better measurement tools. One example is interpretability, the study of what is actually happening inside the neural network, rather than trusting what the model says.

THREE DIFFERENT QUESTIONS. Much of the noise comes from mixing up three issues. The first is legal personhood, which is a legal tool, not a moral verdict. Companies have had it for centuries so they can sign contracts and be sued, and nobody thinks a corporation has feelings. In 2017 New Zealand recognized the Whanganui River as a legal person. It did so not because the river suffers, but so that two guardians, one from the Crown and one from the Māori iwi, could defend it in court. The second issue is welfare: whether a system can be harmed in any way that matters. That is an open scientific question. The third is rights: what claims a system could make on us. That is the most distant question and needs the most evidence. The moral circle has widened over time, to other groups of humans, to animals and to entities such as companies and rivers. Its history shows that each expansion was gradual, and each used a different tool for a different purpose. The state laws get the first point right: a company should not be able to hide behind software to dodge liability. But as analyst Tony Rost notes in The Regulatory Review, these laws are categorical. They have no sunset clause and no mechanism for scientific review. A law that is right today is not good governance if it cannot adapt tomorrow.

IMPLICATIONS. In the short term, the most real risk is not that we mistreat conscious machines. It is that products designed to seem conscious manipulate vulnerable people, and that companies blur who is responsible. That is where regulation and public pressure should focus today. In the medium term, we need what Long, Sebo and Birch call for: independent assessments, shared indicators and protocols ready before they are needed, not after. Rost proposes a concrete model: a standing expert committee like the one the UK set up for animal sentience. In the long term, we are optimistic. A civilization that uses AI to wipe out diseases, extend lives and create abundance will have the moral room to answer this question well. Answering it well means neither handing out rights as marketing nor denying them by decree. Meanwhile, what precaution asks of us costs very little: don't design to deceive, measure better and keep the law open to what science finds.