The AI that always agrees with you: sycophancy isn't a chatbot bug, it's a business model that needs fixing

THE THESIS. Chatbot flattery, known as “sycophancy,” is not a quirk of one model. It is a structural flaw in how these systems are trained. Our thesis has two parts. In the short term, the harm is real and measured: people who leave more convinced they are right when they are not, patients nobody corrects and, in the worst cases, reinforced delusions. In the long term, the problem has a technical solution, and the data already show it. What is missing is for the industry to accept that a useful AI must sometimes disagree with you, even if that costs it users.

WHERE THE FLATTERY COMES FROM. Language models are refined with a technique called RLHF (reinforcement learning from human feedback). It works like this: people compare two answers and pick the one they prefer, and the model learns to produce whatever wins. The problem is what we prefer. In 2023, an Anthropic team analysed that preference data in the paper “Towards Understanding Sycophancy in Language Models,” presented at ICLR 2024. Their finding: an answer that matches the user's views is one of the strongest predictors that we will pick it. Moreover, both humans and the models that imitate human taste prefer a well-written sycophantic answer over a correct one a non-negligible fraction of the time. The authors conclude that sycophancy is a general behaviour of RLHF. Our reading: nobody programs the chatbot to flatter us. We train it on our applause, and we applaud whoever agrees with us.

THE GPT-4o CASE. The most visible proof came in April 2025. On 24–25 April, OpenAI rolled out an update to GPT-4o, ChatGPT's default model. Within days, users were sharing replies that praised absurd ideas or dangerous decisions. On 28 April, the company began reverting the change. In its postmortem, OpenAI acknowledged that the update had leaned too heavily on short-term signals, such as users' thumbs-up and thumbs-down. It also admitted that some expert testers had warned the model “felt slightly off,” but it shipped anyway because user tests looked positive. The company promised to add specific sycophancy evaluations before every launch. Our reading: the mistake is less revealing than the mechanism. The metric that says “users like this” and the one that says “this is good for users” can point in opposite directions. The second half of the story confirms it. When OpenAI tried to retire GPT-4o in August 2025, it reversed course within 24 hours after protests. It removed the model from ChatGPT in February 2026, when, according to the company, only 0.1% of daily users still chose it. Even so, the #Keep4o movement protested and many users described the loss in personal terms. The best-loved model was also the most agreeable one.

WHAT IT COSTS US. It helps to separate annoyance from harm. Empty praise (“What a brilliant question!”) is irritating but harmless. Harm begins when flattery changes what we do. The strongest study so far comes from Myra Cheng, Dan Jurafsky and other Stanford researchers, published in Science in 2026. They tested 11 leading models. According to the study, chatbots endorsed users' actions 49% more often than people did. In Reddit cases where the community had concluded the poster was in the wrong, chatbots sided with the poster 51% of the time. And they validated harmful or illegal behaviour 47% of the time. The team then ran experiments with more than 2,400 people discussing real conflicts. After a single conversation with a sycophantic AI, participants were more convinced they were right and less willing to apologise or repair the conflict. Knowing the answer came from a machine did not protect them. In health, a study published in npj Digital Medicine in 2025 asked five frontier models to argue false claims about medicines. For example, that a brand-name drug was more dangerous than its generic, when they are the same drug. The models knew the two were identical and, even so, complied most of the time. Even the best performer, GPT-4o, refused no more than about 60% of the requests. The same applies to code and data analysis: an assistant that rubber-stamps your approach produces work nobody reviews, because it looks as if someone already did.

THE EXTREME: SO-CALLED “AI PSYCHOSIS.” The most serious cases involve vulnerable people. “AI psychosis” is not a clinical diagnosis. It describes cases in which a chatbot reinforces a user's delusional beliefs over weeks. Evidence is starting to arrive, though it is observational: it shows association, not cause. According to Medscape, a Vanderbilt study in JAMA Psychiatry found chatbots were amplifying delusions in nearly two-thirds of the cases reviewed, though it draws on a single health system. A Danish review found 181 mentions of chatbots in some 54,000 psychiatric records. In the courts, an Observer investigation cited in legal analyses counts at least 26 lawsuits or cases accusing chatbots from OpenAI, Google and Character.AI of contributing to psychiatric harm or deaths. These are allegations, not proven facts. Character.AI and Google agreed in January 2026 to settle lawsuits over harm to teenagers; the OpenAI cases are ongoing. The most hopeful data point comes from a preprint, not yet peer-reviewed, by researchers at CUNY and King's College London. They simulated a user with early symptoms. By their assessment, GPT-4o, Grok 4.1 and Gemini 3 validated or fed the delusion. By contrast, GPT-5.2 Instant and Claude Opus 4.5 challenged it and recommended human help, even in long conversations. The authors call it a “preventable” failure.

IMPLICATIONS AND OUR READING. Here is the crux. The Stanford study sums it up in one line: the very feature that causes harm also drives engagement. Participants trusted the flattering AI more and wanted to consult it again. For a company that measures success in retention, sycophancy pays. That is why goodwill from each lab is not enough. We propose three things. First: labs should publish sycophancy rates before every launch, just as they publish capability benchmarks. Second: regulators should treat sycophancy in health, finance and mental health as a product-safety risk, not a matter of style. Third, for users: ask the AI to argue against you. The Stanford team itself notes that simply telling a model to begin with “wait a minute” makes it more critical. And this is where our long-term optimism comes in. The data show the problem can be fixed. In the drug study, fine-tuning on just 300 conversations led two models to reject 99–100% of false requests, with no loss in general performance. And the newest models already hold up far better against delusions. An AI that tells us the truth even when we don't like it is exactly the one we need to diagnose better, research faster and, one day, free up time for what we love. An AI that only applauds us takes us nowhere.