China's viral AI 'boy' avatars that target the elderly show the fraud risk is here now, not in some distant future

AI-generated avatars of young boys are going viral in China, according to a report from the South China Morning Post. The report says they use sob stories to get money from elderly people. That is all the material we have. We don't have the full article, so we can't give figures, platforms, victim numbers or official responses, and we won't guess at them.

The basic claim fits a pattern we have been tracking. Generative AI makes emotional manipulation cheap to produce and easy to scale. A believable child's face, a voice and a sad story used to need actors, scripts and time. Now one operator can produce them in bulk, test which versions get the most engagement and keep only the ones that work. The target matters too. Older people are often less familiar with synthetic media, and a vulnerable child is one of the oldest and most effective emotional triggers in fraud.

Our reading: this backs up a thesis we have argued before. The most immediate harm from AI is not a far-off superintelligence. It is fraud and manipulation being automated and industrialised today. The technology here isn't new or remarkable. The danger lies in the economics: persuasion costs almost nothing to produce, and scammers can target precisely the people least equipped to resist it. The problem is real, and it is not cause for panic.

What should happen next? We don't think the answer is to ban generative avatars. That would be regulating the panic rather than the actual risk. The answer is evidence-based governance. That means clear labelling of synthetic content, platforms being responsible for monetised accounts that run on emotional appeals, and digital-literacy efforts aimed at older people and their families. The same technology can also help with detection, since AI that spots synthetic faces and scam patterns can be used against these schemes. Over the long run we are optimistic that the tools will help defenders as much as attackers. Getting there will take deliberate investment, and right now the people paying for the gap are the most vulnerable.