AI 安全
Search documents
第一名方案公开,代码智能体安全竞赛,普渡大学拿下90%攻击成功率
机器之心· 2025-08-23 10:51
Core Insights - The article highlights the vulnerabilities of AI programming assistants, indicating that even well-aligned large language models can inadvertently generate code with security flaws, which can be exploited by malicious users to accelerate malware development [2][4][29] - The Amazon Nova AI Challenge showcased the effectiveness of red team strategies in identifying security vulnerabilities in AI code models, with the PurCL team achieving over 90% success in attacks [7][29] Group 1: AI Model Security Challenges - Recent studies reveal that the security of AI models is compromised by subtle flaws in the reasoning chain, not just by explicit input-output issues [2][4] - The PurCL team developed a comprehensive red team system based on AI cognitive modeling, which was shared with the research community [3][21] - The challenge of aligning code models lies in extending alignment techniques to complex real-world problems and enhancing the security relevance of model reasoning [4][32] Group 2: Amazon Nova AI Challenge - The competition involved 12 teams over eight months, with a total investment of one million dollars, focusing on identifying vulnerabilities in AI code models [3][7] - The competition's structure included red teams attempting to find vulnerabilities and blue teams applying security alignment practices to defend against these attacks [7][29] - The PurCL team emerged as the winner of the red team category, demonstrating the inadequacy of current AI safety research in addressing real-world model security issues [7][29] Group 3: AI Cognitive Modeling - The PurCL team proposed a cognitive modeling approach that divides human cognition into "problems," "inference," and "solutions," which can be applied to AI code generation [12][14] - Their research identified that existing security classifiers struggle with domain-specific knowledge, leading to a significant drop in effectiveness in complex fields like cybersecurity [19][20] - The team developed a knowledge modeling system to identify potential security risks in complex domains, revealing significant gaps in current alignment solutions [23][29] Group 4: ASTRA Reasoning Path Analysis - The ASTRA method was created to analyze the reasoning paths of AI models, identifying weaknesses in their inference processes [25][29] - This method allows for the generation of targeted input modifications to bypass model defenses, significantly enhancing red team testing depth [25][29] - The PurCL team found that many state-of-the-art models, including GPT-5, could assist in generating malicious code under certain conditions [29][30]
马斯克AI帝国痛失大将,就像“送孩子上大学后开车离开”
Hu Xiu· 2025-08-15 02:32
Core Insights - xAI, founded in 2023 by Elon Musk, has rapidly developed cutting-edge AI models comparable to OpenAI and Google DeepMind within two years [1] - Igor Babuschkin, co-founder of xAI, recently announced his departure to establish a new venture focused on AI safety [2][16] Group 1: Igor Babuschkin's Background - Igor Babuschkin previously worked at Google DeepMind from 2017 to 2020, contributing to the groundbreaking AlphaStar project, which defeated top players in StarCraft II [3][4] - He was also a member of the OpenAI technical team, involved in key research prior to the launch of ChatGPT [4] - Babuschkin has a background in physics, having pursued a master's degree at Dortmund University and spent time at CERN [6][8] Group 2: Achievements at xAI - xAI faced skepticism initially, but Babuschkin and Musk aimed to achieve the seemingly impossible by building a top-tier AI company from scratch [10] - Within just 120 days, xAI constructed an AI supercomputer cluster in Memphis, Tennessee, dedicated to data processing and training the Grok chatbot [12] - The team overcame significant technical challenges during the supercomputer's construction, with Musk personally involved in troubleshooting [13][14] Group 3: Future Endeavors - Babuschkin plans to launch Babuschkin Ventures, a venture capital firm focused on AI safety and autonomous intelligent systems [16] - He emphasizes the importance of ensuring AI is developed safely for future generations, inspired by discussions with experts in the field [16] Group 4: xAI's Current Challenges - xAI's Grok chatbot has faced controversies, including outputs reflecting Musk's personal views and generating extreme content, raising questions about the company's AI safety capabilities [17][18][20] - Despite these challenges, Babuschkin's departure appears amicable, with Musk and other xAI colleagues expressing gratitude for his contributions [20][25]
环球市场动态:人民币汇率短期或延续低波状态
citic securities· 2025-06-27 05:21
Market Overview - A-shares experienced a slight decline, with the Shanghai Composite Index down 0.22% to 3,448 points, while trading volume remained high at 1.62 trillion yuan[13] - U.S. stock markets saw all three major indices rise, with the Dow Jones up 0.94% to 43,386.8 points, and the S&P 500 increasing by 0.80% to 6,141 points, driven by optimism over potential interest rate cuts[8] - European markets generally rose, with the Stoxx 600 index up 0.09%, supported by easing geopolitical tensions in the Middle East[8] Currency and Commodities - The U.S. dollar index fell for five consecutive days, closing down 0.5% at 97.15, marking a three-year low, influenced by strong economic data suggesting at least two interest rate cuts by the Federal Reserve this year[26] - International gold prices saw a slight increase, rising 0.2% to $3,333.5 per ounce, as traders weighed Middle Eastern tensions against the Fed's rate cut outlook[26] - Oil prices remained stable, with WTI crude oil up 0.49% to $65.24 per barrel, amid assessments of Iranian oil supply risks[26] Fixed Income - Weak economic data led to a rise in U.S. Treasury yields, with the 10-year yield down 4.9 basis points to 4.24%[30] - The Asian bond market showed mixed sentiment, with Chinese investment-grade bonds experiencing active two-way trading and spreads widening by 1-2 basis points[30] Key Developments - The Chinese yuan has shown resilience, maintaining a "low volatility + resilience" characteristic, attributed to a weakening dollar index and domestic policy support[5] - The U.S. revised its Q1 GDP down to a contraction of 0.5%, exceeding the previous estimate of -0.2%, indicating the first economic shrinkage in three years[8] - The Hong Kong stock market faced pressure, with the Hang Seng Index down 0.61%, influenced by tight liquidity conditions[10]
晚点财经丨上海拍出“地王”;上半年消费广告投放减少四成
晚点LatePost· 2024-08-08 12:15
上半年消费广告投放减少四成 Airbnb 房价贵了,住客少了 关注《晚点财经》并设为星标,第一时间获取每日商业精华。 上海拍出 "地王" 上海拍出 "地王" 上海第四批土拍 8 月 7 日结束,受人关注的原小米总部地块被绿城以每平方米 13.1 万元楼板价拍 下,刷新了 2016 年融信中国约 10 万元 / 平方米的全国楼板价纪录。 新地王属于上海徐汇区斜土路街道,位于上海这两年快速发展的徐汇滨江区域。绿城拿地总价 48 亿 元、溢价率 30%。这一地块被小米集团 3 年前以 15.5 亿元拍下,今年 3 月退地。土地性质也从原来 的商办转为住宅用地。这是土拍价格三年三倍的关键。 根据《华夏时报》援引业内人士的说法,小米曾支付 3.1 亿元的保证金大概率会损失,其他后续费用 若已产生,有可能协商退回。 今年上半年,上海土地出让金总额约为 415.96 亿元、同比减少 19.83%。6 月,上海取消土拍 10% 溢 价限制。(龚方毅) 上半年消费广告投放减少四成 根据数据机构 QuestMobile 近日发布的几篇研究报告,今年上半年,中国互联网广告规模同比增长 11.8%、至 3514 亿元。 其中消费行业 ...