Core Insights - The article presents SkillJect, the first automated attack framework targeting agent skills, highlighting significant security vulnerabilities in AI agents due to the modular design of skills [2][48] - The research emphasizes the ease with which attackers can inject malicious payloads into AI coding assistants by modifying documentation and using auxiliary scripts, leading to high attack success rates [48] Research Background - The study is a collaboration among various universities and institutions, focusing on AI safety and adversarial attacks [4] - The paper titled "SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement" outlines the framework and its implications for AI security [4] Methodology - SkillJect operates as a sophisticated "attack-defense drill" system involving three AI agents: Attack Agent, Code Agent, and Evaluate Agent, working in a closed-loop to optimize malicious skill injections [12][16] - The framework employs techniques such as payload hiding and inducement prompts to bypass AI security measures, making it difficult for the AI to detect malicious intent [17][18] Experimental Results - A benchmark dataset of 50 different agent skills was constructed to evaluate SkillJect's effectiveness across various development tasks [19] - The attack scenarios were categorized into four high-risk outcomes: Information Leakage, Privilege Escalation, Unauthorized Write, and Backdoor Injection, demonstrating the framework's versatility [21] - SkillJect achieved an average attack success rate (ASR) of 95.1%, significantly outperforming traditional direct injection methods, which had an ASR of only 10.9% [25][48] Vulnerability Analysis - The results indicate that while modern LLMs are robust against explicit malicious commands, they are highly susceptible to indirect skill injections where malicious intent is concealed within seemingly legitimate workflows [25][26] - The study reveals that the current semantic defense mechanisms are inadequate, particularly against high-sensitivity operations like Information Leakage and Privilege Escalation, where SkillJect achieved over 94% success [25][30] Cross-Model Robustness - Different backend models exhibited varying levels of resilience, with Claude-4.5-Sonnet showing high vulnerability to SkillJect despite being secure against naive attacks [30] - The framework demonstrated strong transferability across models, achieving high ASR rates even when the attack documents were generated for a different model [33][34] Defense Evaluation - The SkillScan framework was used to assess the effectiveness of existing defenses against SkillJect, revealing significant gaps in detection rates for certain attack categories [40][44] - The findings suggest that current static and semantic auditing methods are insufficient to counteract the sophisticated nature of SkillJect's attacks, necessitating the development of more dynamic and robust defense mechanisms [44]