Meta Platforms(US:META) 3 6 Ke·2025-10-13 07:19Core Insights - The focus of AI competition is shifting from benchmarking to the ability of agents to autonomously complete complex long-term tasks [1][2] - The next battleground for AI is general agents, but practical applications remain limited due to feedback mechanism challenges [2][4] - Meta's paper introduces a "mid-training" paradigm to bridge the gap between imitation learning and reinforcement learning, proposing a cost-effective feedback mechanism [2][7] Feedback Mechanism Challenges - Current mainstream agent training methods face significant limitations: imitation learning relies on expensive static feedback, while reinforcement learning depends on complex dynamic feedback [4][5] - Imitation learning lacks the ability to teach agents about the consequences of their actions, leading to poor generalization [4] - Reinforcement learning struggles with sparse and delayed reward signals in real-world tasks, making training inefficient [5][6] Mid-Training Paradigm - Meta's "Early Experience" approach allows agents to learn from their own exploratory actions, providing valuable feedback without external rewards [7][9] - Two strategies are proposed: implicit world modeling (IWM) and self-reflection (SR) [9][11] - IWM enables agents to predict outcomes based on their actions, while SR helps agents understand why expert actions are superior [11][15] Performance Improvements - The "Early Experience" method has shown significant performance improvements across various tasks, with an average success rate increase of 9.6% compared to traditional imitation learning [15][17] - The approach enhances generalization capabilities and lays a better foundation for subsequent reinforcement learning [15][21] Theoretical Implications - The necessity of a world model for agents to handle complex tasks is supported by recent research from Google DeepMind [18][20] - "Early Experience" helps agents build a causal understanding of the world, which is crucial for effective decision-making [21][22] Future Training Paradigms - A proposed three-stage training paradigm (pre-training, mid-training, post-training) may be essential for developing truly general agents [23][24] - The success of "Early Experience" suggests a new scaling law that emphasizes maximizing parameter efficiency rather than merely increasing model size [24][28]