1美元Token撬动4800美元收益！AI挑战百万美元级基准，最赚钱的Agent出现了

Core Insights - The article discusses the economic value of AI in completing expert-level tasks, estimating that AI can deliver $480,000 worth of work for a task valued at $1 million, with a cost of only $100 in tokens [1][17] - The $OneMillion-Bench was developed to measure AI's performance in terms of economic value, using real-world expert tasks across various fields [4][11] Group 1: $OneMillion-Bench Overview - $OneMillion-Bench includes 400 challenging tasks across five major fields: finance, law, healthcare, natural sciences, and industry, with tasks designed to reflect real-world scenarios [4][8] - The benchmark aims to provide a high economic value and differentiation, allowing for automated assessment of AI models [4][11] - Each task is assigned an economic value based on the time and hourly wage of expert completion, totaling over $1 million [8][9] Group 2: Task Design and Evaluation - The benchmark incorporates a diverse range of real-world expert tasks, with each task designed to assess specific decision-making abilities in realistic scenarios [12][14] - A non-linear scoring system is used to evaluate model performance, where positive scoring is conservative, and significant penalties are applied for critical errors [13][11] - The tasks are categorized into 92 subfields, with separate assessments for Chinese and global contexts to accurately reflect regional differences [14][11] Group 3: Model Performance Insights - The strongest models currently achieve a pass rate of over 40%, indicating that while AI can deliver substantial value, it still has a way to go before being fully reliable for professional tasks [17][19] - The average scores of top models suggest they are capable of covering many key points, but the pass rate indicates that fewer than half of the tasks meet the required standards for delivery [19][20] - Complex reasoning remains a challenge for AI models, with difficulties in providing executable details in tasks requiring deep understanding and multi-step reasoning [23][19] Group 4: Future Directions - The article emphasizes the need for AI to enhance its deliverable value and ensure that this value is stable, verifiable, and controllable to translate improvements into productivity and revenue [25][26] - The significance of $OneMillion-Bench lies in quantifying the capabilities of "digital employees," helping to determine which tasks can be confidently assigned to AI [26]