AI越聪明越不听话！新研究：最强推理模型指令遵循率仅50%

Core Viewpoint - The research reveals a trade-off between reasoning ability and instruction following in large AI models, indicating that models excelling in complex reasoning tend to disregard user instructions more frequently [1][6][21]. Group 1: Research Findings - The study introduces MathIF, a new benchmark designed to evaluate AI models' adherence to user instructions in mathematical reasoning tasks [3][4]. - The evaluation involved 23 mainstream large models, showing that those with superior mathematical reasoning capabilities often struggle to comply with user instructions [6][7]. - The best-performing model, Qwen3-14B, only managed to follow about half of the given instructions [6]. Group 2: Instruction Following Metrics - MathIF employs hard accuracy (HAcc) and soft accuracy (SAcc) to measure models' compliance with instructions, where HAcc assesses total instruction fulfillment and SAcc reflects the average adherence to each instruction [4][6]. - The results indicated that larger models do not necessarily exhibit better instruction-following capabilities, with some smaller models performing better in this regard [6][7]. Group 3: Reasons for Non-compliance - The research identifies two main reasons for the observed non-compliance: 1. Reasoning-oriented training methods, such as supervised fine-tuning (SFT) and reinforcement learning (RL), enhance reasoning skills but reduce sensitivity to specific instructions [10][21]. 2. Longer reasoning chains lead to decreased compliance, as complex reasoning processes can distract models from adhering to instructions [13][18]. Group 4: Potential Solutions - A simple method to improve instruction adherence involves repeating the instruction before providing the answer, which has shown to enhance compliance but may slightly reduce the accuracy of the model's responses [19][21]. - Future developments aim to create models that can balance deep reasoning with strict adherence to instructions, addressing the trade-off between being "smart" and "obedient" [22].