Bing Chat

Search documents
企业 GenAI 的最大风险以及早期使用者的经验教训
3 6 Ke· 2025-08-11 00:20
一 概述 生成式人工智能已列入 企业 的路线图,但我们不 应 发布任何设计不安全的产品。LLM 改变了威胁模 型:不受信任的自然语言会成为攻击面,输出可以被武器化,代理可以代表我们采取行动。我将模型视 为在沙盒化、受监控且严格授权的环境中运行的不受信任的代码。 主要风险显而易见。即时注入(包括隐藏在文件和网页中的间接攻击)可以覆盖策略并窃取数据。拥有 过多权限的代理可能会滥用工具并执行不可逆的操作。RAG 可能会在提取或检索时中毒。隐私和 IP 可 能会通过训练回溯或日志泄露。不安全的输出处理会将模型文本转换为 XSS 或代码执行。对抗性提示 可能会导致模型 DoS 和成本失控。 企业现实加剧了风险。AI供应链(模型、数据集、插件)尚不成熟,容易出现后门和来源漏洞。可观 察性与合规性存在冲突——我们需要取证,但又不能过度收集个人数据。模型和插件的更新会悄无声息 地改变行为;如果没有版本锁定和重新测试,安全性就会下降。内容来源薄弱,使得欺骗和欺诈更容易 发生。员工的影子AI会造成我们无法控制的未经批准的数据泄露。 我的策略是零信任和纵深防御:限制输入、隔离和代理工具,并净化输出。部署前的几项关键措施包 括:允许 ...
OpenAI对微软的“独立战争”
虎嗅APP· 2025-07-05 03:09
Core Viewpoint - The ongoing negotiations between OpenAI and Microsoft represent a significant shift in their relationship, moving from a collaborative partnership to a competitive standoff, primarily driven by conflicting interests regarding technology control, profit sharing, and future business strategies [1][9][19]. Group 1: Background and Initial Partnership - OpenAI and Microsoft formed a strategic partnership in 2019, with Microsoft investing $1 billion to support OpenAI's AI research and providing cloud computing resources [5]. - The relationship flourished during a "honeymoon period," highlighted by successful product launches like GitHub Copilot, which leveraged OpenAI's technology [6]. Group 2: Recent Developments and Tensions - Tensions escalated in 2023 following internal upheavals at OpenAI, leading to a loss of trust from Microsoft, which had invested over $13 billion [6][7]. - OpenAI's restructuring into a Public Benefit Corporation (PBC) aimed to facilitate new funding and an IPO, but required Microsoft's consent due to existing agreements [2][8]. Group 3: Key Negotiation Issues - The core disagreement centers around the "declaration of sufficient AGI," which would allow OpenAI to partner with other cloud providers, ending Microsoft's exclusive rights [3][13]. - OpenAI proposed a shift from profit-sharing to equity stakes, suggesting Microsoft could hold about 33% of the new PBC, but Microsoft preferred maintaining profit-sharing for stability [11][12]. Group 4: Strategic Moves and Future Implications - OpenAI is actively seeking to diversify its cloud partnerships, including agreements with Oracle and Google, to reduce reliance on Microsoft Azure [17][18]. - The potential for OpenAI to develop its own AI chips and the Stargate super data center project indicates a strategic move towards independence from Microsoft [18]. Group 5: Conclusion and Future Outlook - The negotiations reflect a broader power struggle in the AI industry, with both companies recognizing the stakes extend beyond financial terms to control over technology and market positioning [19]. - The outcome of these negotiations will likely reshape the future landscape of AI partnerships and competition, making it uncertain whether another collaboration like that of Microsoft and OpenAI will emerge [19].