Core Viewpoint - The article discusses a new paradigm in AI called Nested Learning (NL), which addresses the issue of catastrophic forgetting in large language models and proposes a more efficient learning structure that mimics human cognitive processes [2][10][25]. Summary by Sections Nested Learning Concept - Nested Learning transforms models from a flat computational network to a hierarchical, self-adjusting learning system, inspired by the human brain's memory processes [6][12][14]. - Traditional models like Transformers are seen as simplified versions of NL, lacking the multi-level advantages that NL offers [6][14]. Innovations of Nested Learning - The research team introduced three core innovations based on NL: 1. **Deep Optimizer**: Unlike traditional optimizers, NL's deep optimizer uses a pre-processing mechanism to understand gradient properties and employs MLP neural networks for memory, allowing for flexible parameter adjustments [17][18]. 2. **Self-Modifying Model**: This allows models to autonomously learn how to adjust their parameters during training, adapting to new data without manual intervention [19]. 3. **Continuous Memory System**: Upgrades the traditional short-term/long-term memory structure to a multi-scale memory chain, enabling efficient storage and processing of information [20]. Performance of Hope Model - The Hope model, based on NL, significantly outperforms mainstream baseline models like Transformer, RetNet, and DeltaNet in language modeling and common-sense reasoning tasks, demonstrating lower perplexity and higher accuracy across various metrics [8][23][24]. - For instance, in language modeling tasks, Hope achieved a perplexity of 26.05 with 760M parameters, outperforming other models [24]. Implications of Nested Learning - The introduction of NL represents a paradigm shift in deep learning, moving away from the traditional approach of stacking layers and parameters, and instead leveraging cognitive science to create a collaborative, hierarchical intelligence system [25]. - This new paradigm may enable AI to continuously learn and accumulate knowledge like humans, potentially solving key challenges in long-context reasoning and lifelong learning [25].