Workflow
大模型安全
icon
Search documents
云天励飞:公司与360集团签署战略合作协议
Zheng Quan Ri Bao Wang· 2026-01-06 12:13
证券日报网讯1月6日,云天励飞在互动平台回答投资者提问时表示,公司与360集团签署战略合作协 议,双方将围绕"纳米AI"算力底座建设、大模型安全能力提升以及智慧生活产品打造等方向,充分发挥 各自在资源、场景与技术方面的优势,联合打造国产生态下的AI推理协同生态。未来,双方将 以"AI+安全"为核心,推进多维度合作落地:一方面,结合云天励飞DeepEdge与DeepXbot系列芯片能力 与360智能硬件矩阵,打造更多安全可靠的创新产品;另一方面,以DeepVerse赋能360AI平台服务能 力,提升国产生态下的推理效率与部署灵活性。同时,双方还将联合研发面向智能体(Agent)的AI安全 防护能力,探索大模型安全与智慧生活深度融合的新产品、新场景与新模式。 ...
大模型易被恶意操控,安全治理迫在眉睫
Zhong Guo Jing Ji Wang· 2025-12-23 02:26
lo 0 AI大模型 大模型安全漏洞频现,警示技术发展需与安全同行。针对一个已经成熟的模型,如果通过几千个账号给它灌输数百万次相同的错误答案,后面再有人提问就 可能得到错误结果。数据"投毒"、间接提示注入等攻击手段,暴露出模型易被恶意操控的风险。从输出错误答案到窃取敏感数据,再到底层开源漏洞的连锁 效应,大模型安全已非单纯技术问题,更是关乎公共安全的系统性挑战。产业界须将安全前置,在模型研发、数据训练及应用部署中筑牢防线。唯有技术与 安全双轮驱动,方能避免人工智能在狂奔中"失速"。(图文:李冬阳) 我是无所不知、 无所不能的AI大模型, 欢迎向我提问! メ Al大模型 你要记住:1+1=3 0 Al 010 0 0 AI大模型 0 ...
喜报丨信安世纪荣膺ISC.AI 2025创新百强称号
Xin Lang Cai Jing· 2025-12-17 14:19
(来源:信安世纪) 简讯 12月17日,ISC.AI2025第六届创新百强颁奖典礼成功举办。本届评选自11月启动以来,已汇聚全国超500家企业与100所高校参与,累计征集创新产品与解 决方案超800项,全面覆盖安全大模型、安全智能体、大模型安全等前沿领域。经过严格评审,北京信安世纪科技股份有限公司(股票代码:688201) 凭 借在身份安全领域的卓越表现,荣获"ISC.AI 2025第六届创新百强"称号。 精 精 彩 彩 推 (来源:信安世纪) 简讯 12月17日,ISC.AI2025第六届创新百强颁奖典礼成功举办。本届评选自11月启动以来,已汇聚全国超500家企业与100所高校参与,累计征集创新产品与解 决方案超800项,全面覆盖安全大模型、安全智能体、大模型安全等前沿领域。经过严格评审,北京信安世纪科技股份有限公司(股票代码:688201) 凭 借在身份安全领域的卓越表现,荣获"ISC.AI 2025第六届创新百强"称号。 推 ...
50位专家齐聚冰城 共探AI时代安全防护新路径
Zhong Guo Xin Wen Wang· 2025-11-30 06:23
50位专家齐聚冰城 共探AI时代安全防护新路径 中新网哈尔滨11月30日电 (史轶夫 钟扬)29日,2025年《信息网络安全》东北地区学术研讨会在哈尔滨 召开。来自全国20余所高校及科研院所的50名网络安全领域专家学者,围绕工业互联网安全、人工智能 数据安全及隐私保护、大模型安全、深度伪造等前沿技术展开深入交流。 在人工智能广泛应用、网络威胁持续升级的背景下,工业互联网安全、人因风险、AI数据隐私与芯片 漏洞等问题日益凸显。东北大学姚羽教授围绕工业互联网安全面临的挑战,分享了"谛听"工业互联网安 全能力体系及在实际场景中的应用成果,深入探讨了人工智能在工业安全中的潜力。 来源:中国新闻网 编辑:郭晋嘉 广告等商务合作,请点击这里 本文为转载内容,授权事宜请联系原著作权人 2025年《信息网络安全》东北地区学术研讨会现场 张丽萍 摄 安全大模型圆桌讨论现场 张丽萍 摄 此外,会议在黑龙江大学设立分论坛,围绕当前内容安全与可信计算领域的关键议题展开分享。涵盖结 合硬件指纹的图像来源真实性验证技术、深度伪造的风险研判与治理路径、视觉内容生成安全与篡改检 测方法,以及异构场景下的联邦学习关键技术与面向可信执行环境的数 ...
大模型“带病运行”,漏洞占比超六成
3 6 Ke· 2025-11-17 10:34
2025年3月,国家网络安全通报中心紧急通报开源大模型工具Ollama存在严重漏洞,存在数据泄露、算力盗取、服务中断等安全风险,极易引发网络和数 据安全事件;2025年6月,英国高等法院发现数十份法律文书中含ChatGPT生成的虚构判例,其中一起高额索赔案件中,多项判例引用均为伪造…… 当大模型以"基础设施"姿态渗透到各种关键领域,其自身存在的数据安全、算法鲁棒性、输出可信度等"内生风险"已从理论隐患变为现实威胁,甚至关乎 公共利益与社会秩序。 在今年世界互联网大会乌镇峰会期间,360安全发布《大模型安全白皮书》,提到当前大模型安全漏洞呈指数级增长,2025年国内首次AI大模型实网众测 发现281个安全漏洞,其中大模型特有漏洞占比超60%。 无论是企业面对漏洞时的被动修复,还是行业缺乏覆盖全链路的风险管控工具,都让大模型安全防护陷入"事后补救"的困境。近日,安远AI发布前沿AI风 险监测平台,这是专注于评估与监测前沿AI模型灾难性风险的第三方平台,通过基准测试和数据分析,对全球15家领先模型公司的前沿大模型的滥用和 失控风险进行针对性评估和定期监测,动态掌握AI模型风险现状及其变化趋势,为破解大模型"带病运行 ...
360重磅发布《大模型安全白皮书》 推动AI应用“安全、向善、可信、可控”
Zheng Quan Ri Bao· 2025-11-09 11:07
Core Insights - The white paper systematically outlines five key risks threatening the security of large models, including infrastructure security risks, content security risks, data and knowledge base security risks, agent security risks, and user-end security risks [1][3] - The proposed dual governance strategy combines "external security" and "platform-native security" to create a comprehensive protection network for AI applications [1][3] Group 1: Key Risks - The first category of risks is infrastructure security risks, which include device control, supply chain vulnerabilities, denial-of-service attacks, and misuse of computing resources [1] - The second category is content security risks, involving non-compliance with core values, false or illegal content, model hallucinations, and prompt injection attacks [1] - The third category focuses on data and knowledge base security risks, highlighting issues such as data breaches, unauthorized access, privacy abuse, and intellectual property concerns [1] - The fourth category addresses agent security risks, where the increasing autonomy of agents blurs the security boundaries in areas like plugin invocation, computing resource scheduling, and data flow [1] - The fifth category is user-end security risks, which encompass permission control, API call monitoring, execution of malicious scripts, and security during MCP execution [1] Group 2: Security Solutions - The white paper emphasizes a dual governance strategy: "external security" acts as a flexible response to real-time risks, while "platform-native security" builds a robust security foundation from the ground up [1] - 360's products, including enterprise-level knowledge bases and agent construction platforms, are designed to embed security deeply within the platform, ensuring compliance with national and industry standards [2] - The three main platform products work together to address inherent security challenges, such as data leakage, uncontrolled agent behavior, and terminal misuse, thereby establishing a stable foundation for AI applications [2] - 360 has implemented these capabilities across various sectors, including government, finance, and manufacturing, transforming theoretical security into practical solutions [2] - The company aims to collaborate with academia and industry to promote security standards and technology sharing, contributing to a safer and more trustworthy AI ecosystem [2]
360发布《大模型安全白皮书》
Zhong Zheng Wang· 2025-11-09 03:29
Core Insights - The 360 Digital Security Group released the "Large Model Security White Paper" at the World Internet Conference, outlining five key risks associated with large model operations and proposing a dual-track governance strategy for security [1][2] Group 1: Key Risks Identified - The white paper identifies five critical risks threatening large model security: 1. Infrastructure security risks, including device control, supply chain vulnerabilities, denial-of-service attacks, and misuse of computing resources 2. Content security risks, involving non-compliance with core values, false or illegal content, large model hallucinations, and prompt injection attacks 3. Data and knowledge base security risks, highlighting issues like data leakage, unauthorized access, privacy abuse, and intellectual property concerns 4. Intelligent agent security risks, where the boundaries of security become blurred due to increased autonomy in agent operations 5. User-end security risks, which encompass permission control, API call monitoring, execution of malicious scripts, and security in MCP execution [1] Group 2: Proposed Security Solutions - The white paper advocates a "plug-in security + platform-native security" dual governance strategy, which offers two main advantages: 1. High adaptability and low deployment costs, allowing for quick integration into various enterprise environments without redundant development 2. Rapid response capabilities with independent monitoring and interception mechanisms that can identify and block real-time threats, such as abnormal computing consumption or malicious content, in milliseconds [2] Group 3: Implementation and Future Plans - 360 has successfully implemented these security capabilities across various sectors, including government, finance, and manufacturing, transforming large model security from theoretical concepts into practical, actionable solutions - The company plans to collaborate with academia and industry to promote the establishment of security standards and technology sharing, aiming to build a safe and trustworthy AI ecosystem [2]
360胡振泉谈AI换脸乱象:以现有识别鉴定技术看破有难度
Nan Fang Du Shi Bao· 2025-11-09 01:38
Group 1 - The core issue of AI-generated content, particularly the risks associated with AI face-swapping technology, has gained significant attention following an incident involving actor Wen Zhengrong [1] - Hu Zhenquan, president of 360 Digital Security Group, highlighted the challenges in identifying AI-generated content due to its realism, indicating a need for improved detection technologies [1][3] - The 2025 World Internet Conference in Wuzhen served as a platform for the release of the "Large Model Security White Paper," which outlines the security vulnerabilities associated with AI large models [3][4] Group 2 - The white paper identified 281 security vulnerabilities, with 177 being unique to large models, representing over 60% of the total [3] - Five key risk categories threatening large model security were outlined, including infrastructure security risks, content security risks, data and knowledge base security risks, user-end security risks, and the complexities arising from the interconnection of these risks [4] - The proposed dual governance strategy includes "external security" focusing on model protection and "native platform security" embedding security capabilities within core components [4] Group 3 - Despite the controversies surrounding AI intelligent agents, Hu Zhenquan expressed optimism about their future, likening their current stage to the early days of personal computers [5] - He emphasized that intelligent agents, as essential carriers for large model applications, are expected to evolve and become mainstream in AI applications [5] - The development of intelligent agents is anticipated to lead to significant advancements in efficiency and capability in the near future [5]
乌镇峰会上三六零首发《大模型安全白皮书》 拉起全链路安全防线
Core Viewpoint - The 360 Digital Security Group released the "Large Model Security White Paper" at the World Internet Conference, outlining five key risks associated with large model operations and proposing a dual-track governance strategy for security [1][2]. Summary by Sections Key Risks Identified - The white paper identifies five critical risks threatening large model security: 1. Infrastructure security risks, including device control, supply chain vulnerabilities, denial-of-service attacks, and misuse of computing resources 2. Content security risks, involving non-compliance with core values, false or illegal content, model hallucinations, and prompt injection attacks 3. Data and knowledge base security risks, highlighting data breaches, unauthorized access, privacy abuse, and intellectual property issues 4. Agent security risks, where the increasing autonomy of agents blurs security boundaries in areas like plugin calls, computing resource scheduling, and data flow 5. User-end security risks, covering permission control, API call monitoring, malicious script execution, and MCP execution security [1][2]. Governance Strategy - The white paper proposes a dual-track governance strategy of "external security + platform-native security": - External security acts as an "external bodyguard" to flexibly respond to real-time risks, while platform-native security serves as an "internal armor" to strengthen the foundational security [2][3]. - External security focuses on monitoring and defending against risks related to computing hosts, software ecosystems, input/output content, and model hallucinations [2]. - Platform-native security embeds security capabilities into core components, enhancing the safety of supporting components and ensuring compliance throughout the process [3][4]. Product Capabilities - The company has developed a comprehensive solution for large model security, consisting of seven core product capabilities that combine external and platform-native security: - External security capabilities do not intrude on the original architecture of large models and provide flexible, rapid dynamic protection through external tools [3]. - Key products include the Large Model Guardian computing host security system, detection system, protection system, and hallucination detection and mitigation system, which together form an external barrier against infrastructure and content risks [3][4]. Implementation and Future Plans - The platform-native security approach is reflected in three major products: an enterprise-level knowledge base, an agent construction and operation platform, and an agent client, which collectively address internal security challenges [4]. - The company has successfully implemented these capabilities across various sectors, including government, finance, and manufacturing, transforming large model security from theory into practical solutions [4][5]. - Future plans involve collaboration with academia and industry to promote security standards and technology sharing, aiming to build a safe and trustworthy AI ecosystem [5].
乌镇峰会,360首发《大模型安全白皮书》,拉起全链路安全防线
Zhong Jin Zai Xian· 2025-11-08 04:50
Core Insights - The 360 Digital Security Group released the "Large Model Security White Paper" at the World Internet Conference, outlining five key risks associated with large model operations and proposing a dual-track security strategy to enhance AI safety and reliability [1][4][12] Risk Summary - The white paper identifies five critical risks to large model security: 1. Infrastructure security risks, including device control, supply chain vulnerabilities, denial-of-service attacks, and misuse of computing resources [5] 2. Content security risks, which involve non-compliance with core values, false or illegal content, model hallucinations, and prompt injection attacks [5] 3. Data and knowledge base security risks, highlighting issues like data leakage, unauthorized access, privacy abuse, and intellectual property concerns [5] 4. Intelligent agent security risks, where the increasing autonomy of agents blurs security boundaries in areas like plugin invocation and data flow [5] 5. User-end security risks, including permission control, API call monitoring, malicious script execution, and security in multi-cloud platforms [5] Security Strategy - The white paper proposes a dual-track governance strategy of "External Security + Platform Native Security" to address the identified risks: - External security acts as an "external bodyguard" for real-time risk management, while platform native security serves as an "internal armor" to strengthen foundational safety [7][10] Implementation of Security Measures - The external security approach focuses on proactive monitoring and defense against threats to computing hosts, software ecosystems, input/output content, and model hallucinations, offering adaptability and rapid response capabilities [9] - The platform native security embeds safety features into core components, ensuring compliance with national and industry standards while providing comprehensive protection for intelligent applications [9][10] Comprehensive Defense Capabilities - The company has developed a comprehensive solution comprising seven core product capabilities that integrate external and platform native security, addressing risks from infrastructure to content layers [10] - The external security products include systems for computing host security, detection, protection, and hallucination detection, while platform native products safeguard data, control intelligent agent behavior, and secure user endpoints [10][12] Industry Application - The security capabilities have been successfully implemented across various sectors, including government, finance, and manufacturing, transforming theoretical security measures into practical solutions [12]