突破单链思考上限，清华团队提出原生「并行思考」scale范式

Core Insights - The article discusses the advancements in large language models (LLMs) in complex reasoning tasks, emphasizing the limitations of current sequential reasoning strategies and introducing a new paradigm called "Native Parallel Thinking" to overcome these challenges [2][4][6]. Group 1: Bottlenecks and Challenges - The performance improvement of LLMs has stagnated despite increased computational resources, indicating a scaling bottleneck in sequential reasoning [3][10]. - The phenomenon known as "Tunnel Vision" restricts LLMs to suboptimal reasoning paths once they make an initial flawed decision, making it difficult to correct or discover better solutions later [6][12]. Group 2: Native Parallel Thinking - The research proposes a framework called ParaThinker, which enables LLMs to generate and integrate multiple reasoning paths simultaneously, thus avoiding the "Tunnel Vision" issue and unlocking their potential reasoning capabilities [14][29]. - ParaThinker is designed to train LLMs to explore diverse reasoning paths in a single forward propagation process, leading to higher quality final answers [14][29]. Group 3: Innovations in ParaThinker - ParaThinker incorporates three core innovations: 1. Specialized controllable tokens to guide the model in opening independent thinking paths [18]. 2. Unique thought embeddings to maintain clarity in the source of information during the integration phase [18]. 3. Two-stage attention masks to ensure independence during parallel reasoning and allow global attention during the summarization phase [18]. Group 4: Experimental Results - Experiments show that using 8 parallel paths with a 1.5 billion parameter model leads to an average accuracy improvement of 12.3%, while a 7 billion parameter model shows a 7.5% improvement [23]. - The results indicate that the accuracy increases with the breadth of thinking, demonstrating the effectiveness of parallel reasoning [22][23]. Group 5: Comparison with Majority Voting - ParaThinker can be combined with majority voting strategies to achieve even higher accuracy, showcasing its compatibility with existing methods [26][28]. - The integration of ParaThinker with majority voting allows for a more robust approach to reasoning tasks, enhancing overall performance [26][28].