Workflow
AI智能体安全治理
icon
Search documents
中国电信:2025年AI智能体安全治理白皮书
Sou Hu Cai Jing· 2025-11-27 08:27
Core Insights - The article discusses the transition of AI agents from "conversational intelligence" to "decision-making intelligence," highlighting their capabilities as "digital collaborative partners" with environmental awareness, autonomous planning, and task execution. However, this advancement brings unprecedented security challenges [1][2]. Summary by Sections AI Agent Security Risks - The white paper identifies unique systemic risks associated with AI agents, such as attackers using hidden commands to induce the agent to download malware or employing long messages to trigger privacy breaches. These cases indicate that traditional boundary defenses are ineffective [1]. - Risks span four layers: - **Perception Layer**: Vulnerable to multi-modal input deception and protocol abuse. - **Decision Layer**: Prone to errors due to hallucinations and semantic ambiguities leading to cascading mistakes. - **Memory Layer**: Risks include persistent poisoning and cross-session privilege escalation. - **Execution Layer**: Potential for privilege loss to turn into an attack vector [1][2]. AI Agent Security Governance - To address these challenges, the white paper proposes a dynamic defense system covering the entire lifecycle of AI agents, emphasizing three governance goals: "safe, controllable, and trustworthy." It outlines layered technical countermeasures such as sandbox isolation for tool invocation, dual verification for decisions, encryption and access control for memory content, and real-time blocking of execution behaviors [2]. - The report includes practical case studies from the Starry Sky AI platform, MCP protocol security scanning tools, and automated evaluation of edge AI agents, demonstrating the feasibility and effectiveness of the governance solutions [2]. Call to Action - The white paper serves not only as a technical guide but also as an action plan for building a trustworthy AI ecosystem. It calls for collaboration among government, industry, and enterprises to enhance top-level design, strengthen technical capabilities, and implement differentiated protection strategies for AI agents based on their autonomy levels, ensuring that technological development remains on a secure path [2].