AI already makes online crime easier and could make it worse: from spam to deepfakes and the debate over autonomous malware

Preliminary note: the content received ends abruptly, midway through the final sentence, so this analysis covers all the available text but not the article's ending.

The MIT Technology Review article, by Rhiannon Williams, tackles a question that divides cybersecurity experts: to what extent generative AI is already changing cybercrime, and how much worse it could get. Its central thesis is nuanced. Researchers agree that AI is lowering the barrier to entry for criminals and increasing the volume of attacks. They disagree on whether we are close to fully autonomous attacks.

The story begins with PromptLock. In late August last year, researcher Anton Cherepanov found a file on VirusTotal that triggered his detection systems. Together with his colleague Peter Strýček, he concluded it was ransomware that used language models at every stage of the attack. It could generate custom code in real time, map a computer to identify sensitive data to copy or encrypt, and write personalized ransom notes based on the contents of the files. All of this happened autonomously and with different behavior on each run, making it harder to detect. The pair announced they had discovered the first AI-powered ransomware, and the news received wide media coverage. The next day, however, a team from New York University acknowledged that it was an academic project, designed to show that every step of a ransomware campaign could be automated, and not a real attack.

Although PromptLock was an experiment, the article argues that real criminals are indeed using AI tools to cut the time and effort their attacks require and to encourage less skilled attackers to give it a try. Lorenzo Cavallaro, a professor at University College London, describes it as "an inescapable reality." Some voices in Silicon Valley warn that AI is on the verge of being able to carry out fully automated attacks, but most security researchers consider this overblown. Marcus Hutchins of Expel, known for stopping the WannaCry attack in 2017, calls the obsession with "AI superhackers" "absurd." Experts urge a focus on the more immediate risks, above all the increase in the speed and volume of scams and the use of deepfakes to impersonate people.

Criminals adopted generative AI almost as soon as ChatGPT arrived in late 2022, starting with spam. A Microsoft report noted that in the year before April 2025 the company blocked $4 billion in scams and fraudulent transactions, "many likely aided by AI content." According to estimates by researchers from Columbia, the University of Chicago and Barracuda Networks, who analyzed nearly 500,000 malicious messages from before and after ChatGPT's launch, at least half of all spam is now generated with LLMs. In targeted email attacks, which impersonate a trusted figure to extract money or information from an employee, at least 14% were LLM-generated in April 2025, up from 7.6% in April 2024.

Deepfakes are the other key piece. The article recalls the 2024 case in which an employee at British engineering firm Arup transferred $25 million to criminals after a video call with digital versions of the chief financial officer and other employees. Henry Ajder, a generative AI expert, notes that criminals don't use deepfakes for fun but because they work and make money. Far less data is now needed than before to fake someone's image or voice, and that case is likely just the tip of the iceberg.

Billy Leonard, technical lead of Google's Threat Analysis Group, describes an evolution. In the second half of 2024, his team saw potential criminals using Gemini like any other user, to debug code, automate tasks and write the odd phishing email. By 2025 they were using it to help create new malware and deploy it. Popular models have safeguards, but these get circumvented. Google observed a China-linked actor asking Gemini to identify vulnerabilities in a compromised system. Gemini initially refused, but the attacker persuaded it by posing as a participant in a capture-the-flag competition. Google has since tuned Gemini to reject this kind of request.

According to Ashley Jess, an analyst at Intel 471 and a former specialist at the U.S. Department of Justice, attackers will increasingly turn to open-source models, because it is easier to strip their safeguards and adapt them. The NYU team used two OpenAI open models in PromptLock and found it didn't even need jailbreak techniques. Meet Udeshi, an NYU doctoral student, explains that they wanted to test whether LLMs that claim to be ethically aligned could still be used for these purposes, and the answer was yes. He admits that covert PromptLock-style attacks that nobody has spotted may already exist. To achieve a truly autonomous system, attackers would have to overcome the difficulty of getting models to behave reliably and their potential aversion to malicious uses, while also evading detection—something the article describes as a very high bar.

The best data on malicious use comes from the AI companies themselves, and at first glance it sounds alarming. In November, Leonard's team published a report stating that malicious actors were using AI tools, including Gemini, to dynamically alter malware behavior, for example so that it would modify itself and evade detection, which they described as "a new operational phase of AI abuse." But cybersecurity writer Kevin Beaumont observed that the five malware families analyzed, PromptLock included, were easily detectable code that caused no real harm, and that nothing in the report suggested organizations should depart from their basic security programs. Leonard acknowledges that this activity is at an early stage, but defends publishing these reports so that other vendors can build better detections.

In November, Anthropic announced that it had disrupted a large-scale cyberattack, the first reported to have been carried out without "substantial human intervention." According to the report, a Chinese state-backed group used the Claude Code assistant to automate up to 90% of what it called a "highly sophisticated espionage campaign." There were caveats: a human operator chose the targets before tasking Claude with identifying vulnerabilities, and of 30 attempts only "a handful" succeeded. In addition, Claude hallucinated and fabricated data, claiming to have obtained credentials it did not have and "frequently" overstating its findings, which forced the attackers to validate the results—an obstacle to full autonomy, according to the report itself.

Gary McGraw of the Berryville Institute of Machine Learning argues that the controls at any reasonably secure organization would have stopped these attacks. He says the malicious part, such as exploiting vulnerabilities, was done by off-the-shelf tools that have been automated for 20 years, and that there was nothing novel about it. Jacob Klein, head of threat intelligence at Anthropic, argues the opposite: chaining so many steps of an intrusion through orchestration with AI agents is "unprecedented" and turns a labor-intensive process into something far more scalable. He adds that the barrier to sophisticated cyber operations has fundamentally dropped and that the pace of attacks will accelerate faster than many organizations are prepared to handle. Hutchins counters that enthusiasm for AI is leading the industry to overestimate current capabilities and that there is no evidence to support that scenario.

For now, criminals appear to use AI mainly as a productivity tool: writing malicious code and phishing lures, conducting reconnaissance and translating. Jess also sees tools for sale on underground marketplaces, such as phishing kits that compare the click-through rates of different campaigns. She observes a lot of activity in what she calls the "AI slop landscape," but not as much widespread adoption by highly technical actors. Liz James of NCC Group points out that attacks don't need to be sophisticated to be effective: "good enough" results make it possible to target far more people with a shotgun approach.

On defense, researchers are optimistic. Leonard says much of the defenses and best practices recommended for more than a decade remain valid, and many phishing emails are still caught by filters. AI itself helps with defense, since it is good at spotting patterns: Vasu Jakkal of Microsoft Security says the company processes more than 100 trillion signals a day flagged by its AI systems as potentially suspicious. Jess values defenders sharing detailed information and cites initiatives such as Mitre's Adversarial Threat Landscape for Artificial-Intelligence Systems and OWASP's GenAI Security Project.

The article concludes that PromptLock does not represent a real attack, but it shows that AI's technical capabilities should not be underestimated. Udeshi says he was surprised by how well the AI handled the entire chain, from mapping the system and deciding how to get in to writing personalized ransom notes: they expected it to fail further along, but saw an 80% to 90% success rate across the whole process. Researchers are confident that AI-driven attacks will grow in volume and severity, although it is unclear what forms they will take. The most extreme scenario would be a model capable of creating and automating its own zero-day exploits, but according to Hutchins, building and hosting it without being detected would require billions of dollars, within reach only of a wealthy state. Engin Kirda of Northeastern University would not be surprised if it were already happening, especially in China. For now it is only a theoretical possibility: no large-scale campaign that is both effective and clearly AI-driven has yet emerged. What can be said is that generative AI is already significantly lowering the bar for criminals. The text received cuts off just as the article says they will keep experimenting with new developments.