跳出「黑盒」，人大刘勇团队最新大语言模型理论与机理综述

Core Insights - The article discusses the rapid growth of Large Language Models (LLMs) and the paradigm shift in artificial intelligence, highlighting the paradox of their practical success versus theoretical understanding [2][5][6] - A unified lifecycle-based classification method is proposed to integrate LLM theoretical research into six stages: Data Preparation, Model Preparation, Training, Alignment, Inference, and Evaluation [2][7][10] Group 1: Lifecycle Stages - Data Preparation Stage: Focuses on optimizing data utilization, quantifying data features' impact on model capabilities, and analyzing data mixing strategies, deduplication, and the relationship between memorization and model performance [11][18] - Model Preparation Stage: Evaluates architectural capabilities theoretically, understanding the limits of Transformer structures, and designing new architectures from an optimization perspective [11][21] - Training Stage: Investigates how simple learning objectives can lead to complex emergent capabilities, analyzing the essence of Scaling Laws and the benefits of pre-training [11][24] Group 2: Advanced Theoretical Insights - Alignment Stage: Explores the mathematical feasibility of robust alignment, analyzing the dynamics of Reinforcement Learning from Human Feedback (RLHF) and the challenges of achieving "Superalignment" [11][27] - Inference Stage: Decodes how frozen-weight models simulate learning during testing, analyzing prompt engineering and context learning mechanisms [11][30] - Evaluation Stage: Theoretically defines and measures complex human values, discussing the effectiveness of benchmark tests and the reliability of LLM-as-a-Judge [11][33] Group 3: Challenges and Future Directions - The article identifies frontier challenges such as the mathematical boundaries of safety guarantees, the implications of synthetic data, and the risks associated with data pollution [11][18][24] - It emphasizes the need for a structured roadmap to transition LLM research from engineering heuristics to rigorous scientific discipline, addressing the theoretical gaps that remain [2][35]