Core Insights - The article discusses the rapid growth of Large Language Models (LLMs) and the paradigm shift in artificial intelligence, highlighting the paradox of their practical success versus theoretical understanding [2][5][6] - A unified lifecycle-based classification method is proposed to integrate LLM theoretical research into six stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation [2][7][10] Group 1: Lifecycle Stages - **Data Preparation Stage**: Focuses on optimizing data utilization, quantifying data features' impact on model capabilities, and analyzing data mixing strategies, deduplication, and the relationship between memorization and model performance [11][18] - **Model Preparation Stage**: Evaluates architectural capabilities theoretically, understanding the limits of Transformer structures, and designing new architectures from an optimization perspective [11][21] - **Training Stage**: Investigates how simple learning objectives can lead to complex emergent capabilities, analyzing the essence of Scaling Laws and the benefits of pre-training [11][24] Group 2: Advanced Theoretical Insights - **Alignment Stage**: Explores the mathematical feasibility of robust alignment, analyzing the dynamics of Reinforcement Learning from Human Feedback (RLHF) and the challenges of achieving "Superalignment" [11][27] - **Inference Stage**: Decodes how frozen-weight models simulate learning during testing, analyzing prompt engineering and context learning mechanisms [11][30] - **Evaluation Stage**: Theoretically defines and measures complex human values, discussing the effectiveness of benchmark tests and the reliability of LLM-as-a-Judge [11][33] Group 3: Challenges and Future Directions - The article identifies frontier challenges such as the mathematical boundaries of safety guarantees, the implications of synthetic data, and the risks associated with data pollution [11][18][24] - It emphasizes the need for a structured roadmap to transition LLM research from engineering heuristics to rigorous scientific discipline, addressing the theoretical gaps that remain [2][35]

Core Insights - The article discusses the significant issue of positional bias in language models, which affects their performance in complex reasoning and long-text understanding tasks [1][8] - It introduces Pos2Distill, an innovative "position-to-position" distillation framework designed to transfer the model's strong capabilities from advantageous positions to disadvantaged ones, effectively mitigating positional bias [3][4] Summary by Sections Positional Bias Challenges - Language models exhibit inconsistent sensitivity to different contextual positions, leading to a focus on specific positions in input sequences, which hampers their performance in critical tasks [1] - When comparing two candidate answers, models often favor the first option, compromising their fairness and reliability as evaluators [2] Proposed Solution: Pos2Distill - Pos2Distill aims to leverage the model's acquired knowledge to correct its systematic biases by addressing the performance imbalance caused by positional bias [5] - The framework includes two specialized implementations: Pos2Distill-R1 for retrieval tasks and Pos2Distill-R2 for reasoning tasks, both showing improved consistency across all positions in long-text retrieval and reasoning tasks [5][29] Methodology - The article outlines the distinct behaviors of positional bias in retrieval and reasoning tasks, with retrieval bias manifesting as "token-shifting" and reasoning bias leading to "thought shifting" [10] - Pos2Distill-R1 employs Kullback-Leibler divergence loss to provide fine-grained correction signals for retrieval tasks, while Pos2Distill-R2 uses high-quality chain-of-thought responses from advantageous positions to guide reasoning trajectories [12][13] Experimental Results - Pos2Distill-R1 demonstrated robust and consistent performance, achieving an average accuracy of 56.7% across 20 positions in the WebQ dataset, comparable to the best performance at the optimal "sink position" [22][23] - Pos2Distill-R2 outperformed existing self-training methods, achieving a precise matching score of 42.8 on the MusiQue dataset and 58.3 on the HotpotQA dataset, indicating strong cross-domain generalization capabilities [27][28] Cross-Task Generalization - Both systems exhibit significant generalization capabilities across their respective tasks, with Pos2Distill-R1 enhancing contextual retrieval abilities and Pos2Distill-R2 improving contextual awareness for retrieval tasks [29][30]