At the 2026 Inclusion·Bund Conference, held from September 9 to 12 at the Huangpu Expo Park in Shanghai under the theme "Co-creating the AI New Economy," Alibaba's Chief Scientist, Jingren Zhou, shared insights on AI agent safety. He noted that at the permission level, an agent can be compared to a person; when you issue a command to someone, you simultaneously attach constraints, clearly defining what they are allowed to do.

On the model side, he acknowledged that safety capabilities have improved considerably, but emphasized that substantial work remains to be done. A crucial point, he said, is setting strong constraints on the model and ensuring it consistently adheres to them, without weakening its understanding of those constraints due to reading other material mid-task. He stressed the importance of distinguishing between two categories: hard restriction rules and the data that needs to be processed during task execution. Processing data does not mean the data can override existing constraints. For instance, if a task is set to be completed within 100 yuan, even if the model reads a document stating 200 yuan is required, the agent must still comply with the original budget limit.

Addressing safety and trust issues, he highlighted another key point: agent execution must have a secure sandbox mechanism to prevent scenarios where the intent was to have it do A, but it ended up executing B. This capability is managed by the entire agent system, providing a secure execution environment. He believes the trust issue with agents should not be over-complicated; it is best to treat an agent like a person. Just as when collaborating with colleagues, tasks are assigned with attached constraints. "Collaboration between agents follows a similar logic to collaboration between people. As long as constraints are clearly defined, a sandbox execution environment is built, and the model understands the difference between hard constraints and data to be processed, this issue can be gradually well resolved," he said.