Nvidia(US:NVDA) 机器之心·2025-06-04 04:41Core Insights - The article discusses the potential of Prolonged Reinforcement Learning (ProRL) in enhancing reasoning capabilities in language models, suggesting that it can lead to significant improvements in model performance rather than merely optimizing existing knowledge retrieval [1][15]. Group 1: ProRL Framework - ProRL framework significantly increases the training steps from hundreds to over 2000, unlocking the hidden potential of smaller models [3]. - The framework incorporates a diverse set of verifiable rewards from various domains, providing reliable supervision signals for RL training [5]. - The combination of GRPO and DAPO algorithms enhances training efficiency by avoiding policy update imbalances and filtering ineffective samples [7]. Group 2: Performance Improvements - The Nemotron-Research-Reasoning-Qwen-1.5B model demonstrates remarkable performance across various tasks, outperforming larger models in specific areas [9][10]. - ProRL leads to a 14.7% improvement in mathematical tasks, surpassing 7B models, and a 6.5% lead in code generation over DeepCoder-1.5B [12]. - In logical reasoning, accuracy improves by 54.8%, showcasing the model's enhanced capabilities [12][13]. Group 3: Creativity and Reasoning Expansion - ProRL enables models to solve problems that base models could not, achieving a pass@k of 100% in previously unsolvable tasks [13]. - The training process fosters creativity, allowing models to generate new problem-solving paths rather than relying on rote answers [6][14]. - The longer the training, the stronger the model's ability to deviate from pre-training data, resulting in richer and more creative reasoning strategies [14]. Group 4: Future Implications - The research indicates that ProRL could be the key to developing small language models with strong reasoning capabilities, low deployment costs, and high generalization abilities [16][17].