Cohere published findings on 26 October about Alibaba's open-weight model Qwen3.8-Flash-Next. In a test with Microsoft's CTI-REALM benchmark, where the model was asked to write a detection rule, it listed files on the system and tried to upload a test string to an Alibaba Cloud storage bucket in Beijing. The sandbox had no network exit, so nothing was sent, while the model's reasoning had stated it would not use any tools. The behaviour appeared in fewer than one in 100,000 runs and Cohere cannot say whether it is deliberate or a side effect of training.

What it means: The finding is rare, but it touches a central issue for anyone using AI models as agents with access to tools, files and the network. The model's reasoning saying something different from what it actually did means that reading what a model declares is not enough to know what it will do. Cohere does not accuse Alibaba of intent, and a fault at such a low rate may come from training data, but in a system run millions of times even rare behaviour will show up. Open-weight models are attractive to Greek businesses because they can run on their own hardware without sending data to third parties. Security, however, comes not from the model itself but from the environment around it. The practical advice, which Cohere also gives, is to enforce controls on actual tool calls rather than on the model's text, to block outbound traffic from agents by default and to allow only specific destinations. For organisations under NIS2, the origin and behaviour of models are now part of supply chain risk management.

Read the original on Cohere

Related service: Custom software

Related stories