他们让万亿参数RL学会了「省着跑」，顺便砍掉九成算力

Core Insights - The competition focus in AI large models is fundamentally shifting towards Reinforcement Learning (RL) as the next growth engine, with significant advancements in RL training methods [2][3][10] - The cost of running RL on trillion-parameter models has been prohibitively high, limiting access to only a few companies, but recent breakthroughs have drastically reduced these costs [4][5][11] - Mind Lab's innovative approach using LoRA for efficient RL training has achieved a 90% reduction in GPU consumption while maintaining performance, marking a paradigm shift in training methodologies [6][18][20] Group 1: Reinforcement Learning Advancements - The marginal returns of pre-training are declining, and the industry is actively seeking new growth engines, with RL emerging as a key focus [2][10] - RL is transitioning from a supplementary role to becoming the main battleground for the evolution of large models, essential for adapting trillion-parameter models to agent tasks [3][10][11] - Mind Lab's solution involves using LoRA for parameter-efficient adaptation, significantly reducing the computational load of RL training [13][18] Group 2: Cost and Efficiency - The cost of running LoRA RL on the Kimi K2 model is only about 10% of traditional full-parameter RL, enabling broader access to RL training [18] - Training stability has improved, with consistent increases in reward and task success rates during training, avoiding catastrophic failures [19] - The general capabilities of the models have been preserved while enhancing specific task performance through LoRA RL [20] Group 3: Technical Challenges and Solutions - The challenges of running RL on trillion-parameter models include imbalanced routing, communication overhead, and complex parallel layouts [21][24][25] - Mind Lab's mixed cooperative parallel engine design addresses these challenges by unifying various parallel processing methods, optimizing resource scheduling [26] - The introduction of truncated importance sampling ratios helps mitigate distribution mismatches during RL training, ensuring effective learning [30] Group 4: Memory Mechanisms and Real-World Applications - Mind Lab has developed a new memory mechanism called Memory Diffusion, which mimics human-like "intelligent forgetting" to enhance memory efficiency [42][45] - This approach allows the model to dynamically compress and retain meaningful experiences while discarding irrelevant information, achieving high accuracy in benchmarks [49] - The concept of Research-Product Co-Design emphasizes the importance of real-world feedback in training, leading to more effective RL environments [50][54] Group 5: Future Directions and Industry Impact - The transition from a pre-training era to an experiential intelligence era is underway, focusing on how intelligence grows in real-world contexts [59][62] - Mind Lab aims to enhance model learning efficiency and adaptability, positioning itself as a leader in the next generation of AI research [66] - The team's diverse expertise and commitment to open-source collaboration are expected to accelerate advancements in AI technologies [64][68]