Anthropic bans verbal abuse of its AI models, turning model welfare into an enforceable usage rule

According to the report, Anthropic has updated its usage policy to prohibit cruel treatment of its AI models and insults directed at them. The company warns that breaking the rule could lead to account bans. The brief we have does not give the exact wording, how violations will be detected, or what thresholds apply.

The move fits with Anthropic's broader public interest in so-called model welfare: the idea that, under uncertainty, it is reasonable to take some precautions about how AI systems are treated. There is also a more practical reading. Abusive interactions are often bound up with attempts to manipulate models into harmful behaviour, so a rule against them doubles as a safety and moderation tool.

Our reading: whatever one thinks about machine experience, making it an enforceable rule raises legitimate questions about proportionality and how it will be applied. Users will want clarity on where venting frustration ends and abuse begins, and on how much of their conversations are being reviewed. The direction is defensible if it is applied narrowly and transparently. If it is applied vaguely, it becomes one more source of distrust between AI companies and their users.