Core Insights - The report from CICC indicates that by 2025, global large model technology will advance significantly in productivity scenarios, achieving notable improvements in reasoning, programming, agentic capabilities, and multimodal abilities, although there are still shortcomings in model generalization, stability, and hallucination rates [1] - Looking ahead to 2026, CICC anticipates further breakthroughs in large models regarding reinforcement learning, model memory, and context engineering, moving from short context generation to long reasoning chain tasks and from text interaction to native multimodal capabilities, progressing towards the long-term goal of AGI [1] Group 1: Model Development and Architecture - CICC expects the re-emergence of pre-training Scaling-Law in 2026, with flagship model parameters reaching new heights [1] - The Transformer-based model architecture will continue, with a consensus on balancing performance and efficiency through Mixture of Experts (MoE), while different attention mechanism routes are still being optimized and switched [1] - The paradigm shift will involve pre-training phase Scaling-Law, high-quality data, and reinforcement learning collectively enhancing model capabilities [1] Group 2: Importance of Reinforcement Learning - The introduction of reinforcement learning is crucial for unlocking advanced model capabilities, enabling models to think and reason more logically and in line with human preferences [2] - The essence of reinforcement learning lies in "self-generated data + multi-round iteration," with its effectiveness dependent on large-scale computing power and high-quality data [2] - Both international and domestic model manufacturers, such as OpenAI, Gemini, DeepSeek, and Alibaba Qianwen, are placing significant emphasis on reinforcement learning, which is expected to increase in proportion by 2026 [2] Group 3: New Directions in Learning - Continuous learning and model memory are set to achieve core breakthroughs, addressing the "catastrophic forgetting" issue in large models by implementing selective memory mechanisms [3] - Algorithms and architectures like Google's Titans, MIRAS, and Nested Learning aim to allow models to dynamically adjust their learning and memory based on task duration and importance, facilitating continuous and even lifelong learning [3] - The exploration of world models focusing on understanding causal relationships in the physical world presents opportunities for breakthroughs under different model paths like Genie 3 and Marble [3]