Core Insights - The article discusses the advancements in memory capabilities of large language models (LLMs), highlighting how companies like Google, OpenAI, and Anthropic are integrating memory features into their AI systems to enhance user interaction and continuity in conversations [1][3][10]. Memory Capabilities of LLMs - Google's Gemini has introduced memory capabilities that allow it to retain information across multiple conversations, making interactions more natural and coherent [1]. - OpenAI's ChatGPT has implemented a memory feature since February 2024, enabling users to instruct the model to remember specific details, which improves its performance over time [3][42]. - Anthropic's Claude has also added memory functionality, allowing it to recall previous discussions when prompted by the user [3][6]. Types of Memory in LLMs - Memory can be categorized into sensory memory, short-term memory, and long-term memory, with a focus on long-term memory for LLMs [16][17]. - Contextual memory is a form of short-term memory where relevant information is included in the model's context window [18]. - External memory involves storing information in an external database, allowing for retrieval during interactions, which is a common method for building long-term memory [22][23]. - Parameterized memory attempts to encode information directly into the model's parameters, providing a deeper form of memory [24][29]. Innovations in Memory Systems - New startups are emerging, focusing on memory systems for AI, such as Letta AI's MemGPT and RockAI's Yan 2.0 Preview, which aim to enhance memory capabilities [11][12]. - The concept of hybrid memory systems is gaining traction, combining different types of memory to improve AI's adaptability and performance [37][38]. Notable Memory Implementations - OpenAI's ChatGPT allows users to manage their memory entries, while Anthropic's Claude retrieves past conversations only when requested [42][44]. - Gemini supports user input for memory management, enhancing its ability to remember user preferences [45]. - The M3-Agent developed by ByteDance, Zhejiang University, and Shanghai Jiao Tong University integrates long-term memory capabilities across multiple modalities, including video and audio [10][70]. Future Trends in AI Memory - The future of AI memory is expected to evolve towards multi-modal and integrated memory systems, allowing for a more comprehensive understanding of user interactions [97][106]. - There is a growing emphasis on creating memory systems that can autonomously manage and optimize their memory, akin to human cognitive processes [101][106]. - The ultimate goal is to develop AI systems that can exhibit unique personalities and emotional connections through their memory capabilities, potentially leading to the emergence of artificial general intelligence (AGI) [109][110].

"我真的厌倦了看到那些急于求成的科技初创公司，为了讨好风投而在数据上撒谎，还贴上'SOTA'的标签。"有网友吐槽。 事情源于高人气开源智能体记忆项目 Mem0 在今年 4 月底发布的一篇论文。论文中，该项目团队为可扩展的、以记忆为核心的架构 Mem0 提出了增强版 本，并声称在 LOCOMO 上打败了所有人，其中，Mem0 在 "LLM-as-a-Judge" 指标上相较于 OpenAI 提高了 26%。（论文地址： https://arxiv.org/abs/2504.19413） 当地时间 8 月 13 日， 另一个高人气的智能体记忆框架 MemGPT 的创始团队 Letta AI ，其联合创始人兼 CTO Sarah Wooders 对此公开指控： 几个月前，Mem0 发布了 MemGPT 的基准测试数据，并声称在记忆方面达到了 "SOTA" 水平。 奇怪的是，我完全不知道他们到底是怎么跑这个基准测试的，如果不对 MemGPT 做重大修改，这个测试根本没法完成（他们没有回应我们关于实验具体 运行方式的询问）。 arXiv 并不是经过同行评审的平台，所以不幸的是，近年来公司可以随意发布任何他们想要的"研究 ...

整理 | 褚杏娟 "我真的厌倦了看到那些急于求成的科技初创公司，为了讨好风投而在数据上撒谎，还贴上'SOTA'的 标签。"有网友吐槽。 事情源于高人气开源智能体记忆项目 Mem0 在今年 4 月底发布的一篇论文。论文中，该项目团队为 可扩展的、以记忆为核心的架构 Mem0 提出了增强版本，并声称在 LOCOMO 上打败了所有人，其 中 ， Mem0 在 "LLM-as-a-Judge" 指 标 上 相 较 于 OpenAI 提 高 了 26% 。 （ 论 文 地 址 ： https://arxiv.org/abs/2504.19413） 当地时间 8 月 13 日， 另一个高人气的智能体记忆框架 MemGPT 的创始团队 Letta AI ，其联合创始 人兼 CTO Sarah Wooders 对此公开指控： 几个月前，Mem0 发布了 MemGPT 的基准测试数据，并声称在记忆方面达到了 "SOTA" 水 平。 奇怪的是，我完全不知道他们到底是怎么跑这个基准测试的，如果不对 MemGPT 做重大修 改，这个测试根本没法完成（他们没有回应我们关于实验具体运行方式的询问）。 arXiv 并不是经过同行评审的平台 ...

Group 1: Core Insights - The AI memory system is evolving from Retrieval-Augmented Generation (RAG) to a multi-level state dynamic evolution, enabling agents to retain experiences and manage memory dynamically [1][2]. - Various AI memory projects have emerged, transitioning from short-term responses to long-term interactions, thereby enhancing agents with "sustained experience" capabilities [2][3]. - MemoryOS introduces a hierarchical storage architecture that categorizes dialogue memory into short-term, medium-term, and long-term layers, facilitating dynamic migration and updates through FIFO and segmented paging mechanisms [2][3]. - MemGPT adopts an operating system approach, treating fixed-length context as "main memory" and utilizing paging to manage large document analysis and multi-turn conversations [2][3]. - Commercial platforms like ChatGPT Memory operate using RAG, retrieving user-relevant information through vector indexing to enhance memory of user preferences and historical data [2][3]. Group 2: Challenges Facing AI Memory - AI memory systems face several challenges, including static storage limitations, chaotic multi-modal and multi-agent collaboration, retrieval expansion conflicts, and weak privacy control [4][5]. - The need for hierarchical and state filtering mechanisms is critical, as well as the ability to manage enterprise-level multi-tasking and permissions effectively [4][5]. - These challenges not only test the flexibility of the technical architecture but also drive the evolution of memory systems towards being more intelligent, secure, and efficient [4][5].

Core Viewpoint - The article discusses the development of M+, a scalable long-term memory extension framework built on MemoryLLM, which significantly enhances the effective memory span of language models from under 20k tokens to over 160k tokens while maintaining the same GPU memory usage [2][18]. Summary by Sections Background and Motivation - The paper highlights the distinction between context windows and memory, noting that existing memory models have limitations. For instance, models like GPT-4.1, despite supporting up to 1 million tokens, face challenges in local deployment due to increased GPU memory and latency [4][5]. - The industry standard approach, "Token-Level Memory," involves storing historical content in databases or vector stores, which can lead to redundancy, conflict resolution issues, and weak multimodal capabilities [5]. M+ Framework - M+ introduces a long-term memory component to MemoryLLM, allowing for a more human-like information storage method through latent space memory, which is both compressed and end-to-end trainable [6][7]. - The framework incorporates approximately 1.67 billion memory tokens into the 8B Llama3 model, enhancing the model's ability to retain information over longer sequences [8][13]. Memory Management - During the update phase, the last K memory tokens are combined with new information and processed through a transformer, while old tokens are randomly discarded and replaced with new ones [11]. - The design allows for effective memory retention within 50k tokens, with plans to further expand memory capacity beyond the initial 1.67 billion tokens [13]. Retrieval Mechanism - A co-trained retriever is introduced to enhance the extraction capabilities from long-term memory, as initial attempts using attention mechanisms proved limited [16]. - This structure allows the model to achieve an effective memory span of 160k tokens without significantly increasing GPU load, as most memory resides in CPU [18]. Performance and Results - M+ demonstrates superior information retention capabilities on the SQuAD dataset, outperforming previous models and maintaining information even at 160k tokens [20]. - A comparison of GPU memory costs shows M+ to be more efficient than other models, indicating its potential for practical applications [19]. Conclusion - M+ represents a significant advancement in exploring latent space long-term memory, providing a solid technical foundation for future language models with sustained memory capabilities. The company aims to continue researching more efficient storage mechanisms and intelligent retrieval strategies [22].