Core Insights - The paper "SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks" reveals critical truths about the development of AI agents, emphasizing that agents cannot self-teach new skills effectively [2][40] - The research involved 36 scholars from top institutions and tech companies, highlighting the collaborative effort in understanding agent skills [2] Group 1: Agent Skills Overview - Agent Skills are structured knowledge packages that enhance LLM Agents during inference, differing fundamentally from traditional prompts and tools [5] - The Skills ecosystem is experiencing rapid growth, with a total of 84,192 skills created within 136 days, averaging 810 new skills daily [8] - The paper establishes a benchmark for evaluating the effectiveness of these skills, addressing the lack of standard methods in the industry [9] Group 2: Research Design and Methodology - The research design involved three phases: aggregation of skills, quality screening, and evaluation across various conditions and agent models [14][15] - A total of 86 high-quality tasks were selected from 322 candidates, covering 11 domains, ensuring rigorous testing standards [15][18] Group 3: Key Findings - Finding 1: Expert-built skills resulted in a significant average success rate increase of 16.2 percentage points, demonstrating the value of human expertise in skill development [20] - Finding 2: AI-generated skills were found to be ineffective, leading to a decrease in success rates by 1.3 percentage points, challenging the narrative of self-evolving agents [22][23] - Finding 3: The effectiveness of skills varies significantly across domains, with healthcare and manufacturing showing the highest leverage effects [24][26] - Finding 4: Smaller models combined with skills outperform larger models without skills, indicating a shift in strategy towards optimizing skill integration rather than solely focusing on model size [27][29] Group 4: Engineering Insights and Industry Implications - The research indicates that providing 2-3 skills yields peak performance improvements, while excessive skills lead to cognitive overload and diminished returns [31] - A focus on detailed and targeted skills documentation enhances performance, contrasting with comprehensive documents that may hinder effectiveness [32] - The findings suggest a strategic shift in AI development, emphasizing the importance of high-quality vertical skills over merely scaling model parameters [35][36]