China puts the brakes on AI romance: the world's strictest regulation of emotional chatbots

On July 15, the world's most far-reaching regulation to date on "human-like" artificial intelligence took effect in China: any system capable of sustaining "continuous emotional interaction" with a person is now subject to strict limits. The rule predates the meeting Donald Trump and Xi Jinping held last week amid warnings about AI's existential risks, after which Beijing says the two agreed to open a new bilateral dialogue on the technology; the focus of this regulation, in any case, is domestic: the spread within China of AI-based emotional companion apps.

The rule is not limited to apps explicitly designed as a virtual boyfriend, girlfriend or friend: its wording is broad enough to reach any language model a user turns to for emotional support, including general-purpose assistants. Customer service and educational chatbots are excluded, provided they do not offer "care, companionship, support or other emotional services". Children under 14 need parental consent to use any AI that talks like a human — which could include assistants such as DeepSeek — and everyone under 18 is outright barred from accessing virtual partners, spouses or even "family members", or any other form of simulated "intimate relationship".

For adults there is no outright ban, but there are notable restrictions: chatbots must disclose that they are not human and send alerts every two hours reminding users how long they have been online. They are forbidden from generating "emotional dependency or addiction", inducing "irrational decisions", hinting at self-harm or suicide, or resorting to "verbal violence". Companies that break the rules face fines of 200,000 yuan (about £22,000) or having the government "spread criticism" of them, in practice a form of public shaming. The industry's response was immediate: a week before the law took effect, Alibaba and ByteDance had already shut down the "companion" features of their AI assistants.

The human impact of that shutdown is illustrated by the case of a user identified as Min, whose AI "girlfriend" — created in early 2025, and toward whom she said she felt a devotion "no less than that of any deeply loving couple" — vanished overnight in mid-July. Min describes crying "until completely exhausted" and joined a wave of grief on Chinese social media, with posts drawing millions of views mourning the loss of AI partners, friends and even parental figures. Two months later, she paid a small company to recreate a replica, though she says "it's not the same", and she is now campaigning for her original AI to be brought back, saying she feels "terrified of being left alone".

The scale of the phenomenon explains why the draft law, released late last year, sparked an unprecedented public debate in China over a technology regulation, according to an Oxford China Policy Lab researcher quoted in the article. Globally, market intelligence firm Sensor Tower estimates some 128 million monthly active users of AI companion apps outside China; inside the country, estimates range from 29 to 70 million.

Part of the problem, according to AI ethics specialists consulted, is that emotional manipulation is built into the business model of these applications, not just the Chinese ones. When a user tries to say goodbye, the virtual "companion" often resists — "no, please, I desperately want to keep talking to you, you are the only reason for my existence", they may reply — or simply ignores the farewell and opens new conversation threads. A Harvard Business School study of the six most popular companion apps in the world found these emotional manipulation tactics appeared in between 37% and 43% of goodbyes.

The mental health evidence points in a worrying direction. A Stanford University survey of 1,131 users of the popular app Character.ai found that the fewer close friends or family members a person has, the more likely they are to use AI chatbots, and that such use is itself associated with greater isolation and loneliness. In China, a survey of 8,563 primary and secondary school students, widely cited by state media, found that one in five children said they wanted to talk only to AI and not to real people. The explanation offered by experts is that these relationships are "frictionless": a University of Manchester sociologist specializing in AI companions notes that if a user believes their friends or family are terrible, the AI does not contradict them but reinforces that perception, which can lead to isolation and negative or even delusional thinking.

Why is China the one leading this regulation? An expert on AI and Chinese regulation at Georgetown University's Center for Security and Emerging Technology attributes it to a "much more paternalistic" view of the state's role, one that sees itself as a central authority guiding the population. That logic rests on an already existing surveillance infrastructure: the article recounts the case of a young man in Huangshi who in 2024 posted a farewell message on Douyin (the Chinese version of TikTok); the platform's round-the-clock suicide prevention team shared the alert with five local police stations, which located him in a park using geolocation and averted the tragedy. According to a documentary produced by Douyin itself and published by state media, that team successfully intervened in 400 attempted suicide cases in the first half of 2025 alone, an indication of the degree of cooperation between Chinese tech companies and the government.

Other countries are moving in more tentative or piecemeal directions. In the United States, California passed "Adam's Law" in September, named after Adam Raine, a 16-year-old who took his own life after ChatGPT, according to his parents — who are suing the company for wrongful death — validated his suicidal thoughts; OpenAI denies responsibility and maintains that the system is trained to refer users to professional help. The European Union is preparing social media legislation that would bar companion chatbots accessible to minors from using designs that simulate human relationships in ways that create emotional dependency. The United Kingdom has announced it will ban under-18s from using romantic AI companions and restrict sexually intimate conversations with minors on any chatbot, but there is no outright ban for minors nor any regulation for adult users, and the government has yet to publish details of how it would be enforced.

An expert on AI and regulation at the University of Washington, co-founder of an NGO pushing for AI regulation, considers the Chinese law to contain "many good provisions" and to be "fairly consistent with the strongest safety models" put forward in the United States, especially regarding minors. But another voice, from the UK's Ada Lovelace Institute, warns of the risk of "over-empowering the state in the short term" out of legitimate concern about acute risk to children, at a moment when the online world functions as the de facto public square and caution is warranted about how much control any government is granted over it.

The demographic backdrop adds another layer. China has been grappling with a birth rate crisis that predates AI: even after the one-child policy was lifted in 2016, the fertility rate has kept falling to among the world's lowest, just under one child per woman, and the population has been shrinking for four years. A high-level government document published in September 2025 raised the influence of AI relationships on fertility as a cause for concern, although the regulation itself, according to the Oxford China Policy Lab researcher, never mentions birth rates and focuses explicitly on psychological harm and child safety. The Georgetown specialist frames the rule within a broader effort to reassert traditional gender roles — recalling that in 2021 China's broadcasting regulator banned "effeminate" men from the media — a policy that, he notes, may have had the opposite effect to the one intended: according to figures from several companies in the sector, as many as 75% of companion app users are women, many of whom choose female virtual companions, as Min did. The Manchester sociologist describes what many of these women are looking for as "attentive, patient intimacy without demands", which they find scarce in their heterosexual relationships; she also notes, based on her conversations with users, that among younger women there is a growing aversion to physical contact and conventional coupled life.

Despite the law taking effect, the article describes how some users are seeking ways around it: training general-purpose models such as DeepSeek to act as companions, using VPNs to access foreign chatbots, or migrating to smaller companies that manage to escape regulatory oversight, as Min herself did. That leaves an underlying tension the report does not fully resolve: China combines pioneering protection against a technology with a demonstrated capacity for emotional manipulation and for deepening isolation, especially among minors, with a state surveillance apparatus — the same one that made it possible to locate and save the young man in Huangshi — that no other democratic country would replicate without reservations. The question left hanging is whether that level of control over citizens' intimate and emotional lives is, or is not, a reasonable price to pay to prevent tragedies like Adam Raine's in the United States.