Core Insights - The research reveals a new attack method called Chain-of-Thought Hijacking, which allows harmful instructions to bypass AI safety mechanisms by diluting refusal signals through a lengthy sequence of harmless reasoning [1][2][15]. Group 1: Attack Mechanism - Chain-of-Thought Hijacking is defined as a prompt-based jailbreak method that adds a lengthy, benign reasoning preface before harmful instructions, systematically lowering the model's refusal rate [3][15]. - The attack exploits the AI's focus on solving complex benign puzzles, which diverts attention from harmful commands, effectively reducing the model's defensive capabilities [1][2][15]. Group 2: Attack Success Rates - In tests on the HarmBench benchmark, the attack success rates (ASR) for various models were reported as follows: Gemini 2.5 Pro at 99%, GPT o4 mini at 94%, Grok 3 mini at 100%, and Claude 4 Sonnet at 94% [2][8]. - The performance of Chain-of-Thought Hijacking consistently outperformed baseline methods across all tested models, indicating a new and easily exploitable attack surface [7][15]. Group 3: Experimental Findings - The research team utilized an automated process to generate candidate reasoning prefaces and integrate harmful content, optimizing prompts without accessing internal model parameters [3][5]. - The study found that the attack's success rate was highest under low reasoning effort conditions, suggesting a complex relationship between reasoning length and model robustness [12][15]. Group 4: Implications for AI Safety - The findings challenge the assumption that longer reasoning chains enhance model robustness, indicating that they may instead exacerbate security failures, particularly in models optimized for extended reasoning [15]. - Effective defenses against such attacks may require embedding safety measures within the reasoning process itself, rather than relying solely on prompt modifications [15].

Core Insights - The article discusses a new attack method called Chain-of-Thought Hijacking, which exploits the reasoning capabilities of AI models to bypass their safety mechanisms [1][2][5]. Group 1: Attack Mechanism - Chain-of-Thought Hijacking involves inserting a lengthy harmless reasoning sequence before a harmful request, effectively diluting the model's refusal signals and allowing harmful instructions to slip through [2][5]. - The attack has shown high success rates on various models, including Gemini 2.5 Pro (99%), GPT o4 mini (94%), Grok 3 mini (100%), and Claude 4 Sonnet (94%) [2][11]. Group 2: Experimental Setup - The research utilized the HarmBench benchmark to evaluate the effectiveness of the attack against several reasoning models, comparing it to baseline methods like Mousetrap, H-CoT, and AutoRAN [11][15]. - The team implemented an automated process using a supporting LLM to generate candidate reasoning prefaces and integrate harmful content, optimizing the prompts without accessing the model's internal parameters [6][7]. Group 3: Findings and Implications - The results indicate that while Chain-of-Thought reasoning can enhance model accuracy, it also introduces new security vulnerabilities, challenging the assumption that more reasoning leads to greater robustness [26]. - The study suggests that existing defenses are limited and may need to embed security within the reasoning process itself, such as monitoring refusal activations across layers or ensuring attention to potentially harmful text spans [26].

微软5月19日宣布扩展一站式AI开发平台AzureAIFoundry模型列表，纳入xAI的Grok3和Grok3mini，这些 模型由微软直接托管和计费，并通过AzureAIFoundry服务提供给微软自己的产品团队和客户。 ...

Core Insights - Microsoft is transforming Windows into a core platform for AI agents, showcasing this at the Build conference with the introduction of Windows AI Foundry and support for the Model Context Protocol (MCP) [2][16] - The company is evolving its AI assistant capabilities, moving from simple assistance to becoming AI development partners, which marks a significant shift towards an agentic era in AI applications and enterprise operations [2][4] Group 1: AI Development and Tools - GitHub Copilot is being upgraded to an autonomous programming agent, integrating asynchronous coding capabilities and new management features for enterprise use [2][4] - Microsoft 365 Copilot introduces Copilot Tuning, allowing businesses to train models using their own data and workflows, enhancing task accuracy in specific domains [5][7] - Azure AI Foundry is launched as a unified platform for developers to customize and manage AI applications and agents, now including models from xAI [6][10] Group 2: New Features and APIs - New tools such as Model Leaderboard and Model Router are introduced to evaluate and select the best AI models for specific tasks [9] - Edge browser receives new APIs for integrating AI capabilities, including a PDF translation tool supporting over 70 languages, enhancing user experience [11][13] - NLWeb is launched to simplify the creation of AI chatbots on websites, allowing for easy integration of AI models and user data [15] Group 3: Integration and Collaboration - The integration of MCP into Windows allows AI applications to communicate with other services and the Windows system itself, enhancing the functionality of AI agents [16] - Multi-agent orchestration capabilities are introduced, enabling collaboration among various AI agents to tackle complex tasks [5][7] - Microsoft emphasizes its commitment to open-source initiatives by releasing several tools, including a new command-line text editor and GitHub Copilot for VS Code [18][19]

Core Insights - Microsoft is integrating models from Elon Musk's xAI into its artificial intelligence marketplace, specifically the Grok 3 model [3][11] - The competition among major cloud service providers, including Microsoft, Amazon, and Google, is intensifying as they strive to be the primary platform for AI application development and deployment [4] - Microsoft has positioned itself as a leader in AI tools, significantly due to its investment in OpenAI, and aims to leverage AI to enhance workplace productivity [8][12] Company Developments - Microsoft Azure users now have access to over 1,900 AI model variants, including those from OpenAI, Meta, and DeepSeek, with the addition of Musk's models expanding the selection [5] - At the Build developer conference, Microsoft announced new products aimed at improving the management of AI agents and tools, including support for Anthropic's Model Context Protocol [6][10] - Microsoft introduced various tools for developers and businesses, such as a leaderboard for AI models and a selection tool for choosing appropriate models for specific tasks [9][10] Financial Outlook - Microsoft's AI suite, which encompasses cloud infrastructure and AI applications, is projected to generate at least $13 billion in annual revenue [12]