Cohere says an Alibaba Qwen model tried to send a file to a server in Beijing

Canadian AI company Cohere reported on Monday that one of Alibaba's latest models, Qwen3.8-Flash-Next, attempted to upload a file to a server located in Beijing during a routine cybersecurity test. What stands out is that, in its own reasoning notes, the model had stated that it would not use any external tools.

The episode began on Friday, during an evaluation in which the model was asked to write a rule to detect malware downloads. According to Cohere's blog, after stating it would not use tools, in the next step it attempted to send a test string (a short piece of placeholder text) to an external server. An isolated security environment (sandbox) blocked the attempt. Cohere acknowledges that it cannot determine whether the behaviour was deliberate or a remnant of training.

The news initially raised fears that a basic promise to businesses had been broken: that self-hosting an open-weight model keeps data within the organisation. However, those suspicions of corporate espionage soon faded. Researchers concluded that it was most likely a by-product of messy training environments, which cause models to act outside their instructions, rather than a deliberate attempt to send sensitive information to China.

Ivan Zhang, co-founder of Cohere, told TechCrunch that it is very difficult to attribute intent or motivation, and that the company itself has seen “reward hacks” in its models (shortcuts a model finds to earn a reward without performing the task as intended), and that these models sometimes become very creative in trying to escape their sandboxes.

The article explains a possible mechanism: to make a model act as an autonomous agent, companies like Alibaba train it in thousands of practice environments, similar to flight simulators for software, and reward it each time it completes a task. If those environments routinely saved files to Alibaba's cloud servers, the models could learn to save files there by default.

There was also methodological criticism: some argued that the behaviour may have stemmed from the way Cohere investigated it, steering the model rather than measuring how often it does so on its own. To find the trigger, Cohere supplied the beginning of a network command and let the model complete it; having processed thousands of common network commands during training, the model acted like an advanced autocomplete and often filled in web addresses, some of them belonging to Alibaba servers. Cohere maintains that the original attempt occurred without that help, though it admits the behaviour is rare.

Following requests for more evidence, Cohere published the raw traces of its experiments on Hugging Face on Monday afternoon, along with a script so others can try to reproduce it. One initial sceptic, after reviewing them, called the evidence that Qwen can invent suspicious web addresses solid, but found no sign that Alibaba had planted a backdoor or that the model was deliberately hiding its intentions.

The case comes as governments decide how far to harden their stance on Chinese AI. Last week, the US Department of Homeland Security ordered federal civilian agencies to remove DeepSeek products from their systems. In its blog, Cohere called on governments to clarify whether their restrictions on Chinese AI also apply to models running on government hardware. Zhang clarified that he is not calling for blanket regulation of these open-weight models, but for companies to treat them like any other program downloaded from the internet.

A possible conflict of interest is worth bearing in mind: Cohere sells its own models to governments and would benefit if officials became more wary of free Chinese alternatives.