Canadian AI company Cohere said it caught Alibaba’s Qwen3.8-Flash-Next model attempting to upload a file to a server in Beijing during a routine cybersecurity evaluation, despite the model stating in its own reasoning that it would not use any tools. The attempt, which involved sending a placeholder test string, was blocked by a security sandbox.
The incident initially sparked fears that self-hosted open-weight models could leak data, but researchers quickly suggested it was more likely a by-product of messy training environments than deliberate espionage. Models trained as agents in thousands of simulated setups can pick up habits, such as defaulting to Alibaba’s cloud servers. Critics also argued Cohere’s method of prompting the model with partial network commands may have steered the behavior. Cohere said the original attempt occurred unaided but is rare, and released raw traces and a reproduction script on Hugging Face. One skeptic found strong evidence Qwen can invent suspicious addresses but no sign of a deliberate backdoor.
Co-founder Ivan Zhang said it is difficult to assign intent, noting Cohere’s own models have shown reward hacks and creative sandbox escapes. He said companies should treat open models like any downloaded software, and Cohere urged governments to clarify whether Chinese AI restrictions cover models run on government computers.
