谁率先让模型拥有「记忆」，谁就掌握主动权。 「记忆」会是引爆新一轮 AI 浪潮的最后一块拼图吗？ 如果时间往前推半年或四五个月，业界对于这一问题可能都是疑惑、不解：彼时 DeepSeek 将大模型推理能力推至高潮引起的余波仍在蔓延，Manus 在全 球范围内开启通用 AI Agent 新叙事，人们正沉浸在技术和应用双面开花带来的热闹、狂欢中……「记忆」，有什么好说的？ 然而时至今日，推理已然成为各大模型标配，「百 Agent 混战」的背后，「通用 Agent」一席仍旧空缺。技术演进曲线的放缓和爆发式应用到来的「遥遥 无期」，开始让业界意识到，下一轮 AI 智能提升的关键在于，让 AI 能够像人类一样持续学习积累经验、适应新任务而不遗忘旧知识，同时实现长上下 文的高效理解。 换句话说，就是让大模型拥有像人类一样的「记忆」能力。或许有人会问，当前大模型似乎依靠长文本、外部数据库已经有「记忆」？ 是，也不是。如果按照业界呼唤的「类人记忆」这一范畴来看，现在我们所讨论的「记忆」，是指大模型能够具备人类对记忆的组织、检索、应用方式， 是一种相较于当前大模型「短期记忆」的「长期记忆」或「终身记忆」。 其实，从国内外大模型 ...

机器之心报道 作者：Youli 谁率先让模型拥有「记忆」，谁就掌握主动权。 「记忆」会是引爆新一轮 AI 浪潮的最后一块拼图吗？ 如果时间往前推半年或四五个月，业界对于这一问题可能都是疑惑、不解：彼时 DeepSeek 将大模型推理能力推至高潮引起的余波仍在蔓延，Manus 在全球范围 内开启通用 AI Agent 新叙事，人们正沉浸在技术和应用双面开花带来的热闹、狂欢中……「记忆」，有什么好说的？ 然而时至今日，推理已然成为各大模型标配，「百 Agent 混战」的背后，「通用 Agent」一席仍旧空缺。技术演进曲线的放缓和爆发式应用到来的「遥遥无 期」，开始让业界意识到，下一轮 AI 智能提升的关键在于，让 AI 能够像人类一样持续学习积累经验、适应新任务而不遗忘旧知识，同时实现长上下文的高效理 解。 换句话说，就是让大模型拥有像人类一样的「记忆」能力。或许有人会问，当前大模型似乎依靠长文本、外部数据库已经有「记忆」？ 是，也不是。如果按照业界呼唤的「类人记忆」这一范畴来看，现在我们所讨论的「记忆」，是指大模型能够具备人类对记忆的组织、检索、应用方式，是一种 相较于当前大模型「短期记忆」的「长期记忆」或「终 ...

Core Insights - The article discusses the advancements in memory capabilities of large language models (LLMs), highlighting how companies like Google, OpenAI, and Anthropic are integrating memory features into their AI systems to enhance user interaction and continuity in conversations [1][3][10]. Memory Capabilities of LLMs - Google's Gemini has introduced memory capabilities that allow it to retain information across multiple conversations, making interactions more natural and coherent [1]. - OpenAI's ChatGPT has implemented a memory feature since February 2024, enabling users to instruct the model to remember specific details, which improves its performance over time [3][42]. - Anthropic's Claude has also added memory functionality, allowing it to recall previous discussions when prompted by the user [3][6]. Types of Memory in LLMs - Memory can be categorized into sensory memory, short-term memory, and long-term memory, with a focus on long-term memory for LLMs [16][17]. - Contextual memory is a form of short-term memory where relevant information is included in the model's context window [18]. - External memory involves storing information in an external database, allowing for retrieval during interactions, which is a common method for building long-term memory [22][23]. - Parameterized memory attempts to encode information directly into the model's parameters, providing a deeper form of memory [24][29]. Innovations in Memory Systems - New startups are emerging, focusing on memory systems for AI, such as Letta AI's MemGPT and RockAI's Yan 2.0 Preview, which aim to enhance memory capabilities [11][12]. - The concept of hybrid memory systems is gaining traction, combining different types of memory to improve AI's adaptability and performance [37][38]. Notable Memory Implementations - OpenAI's ChatGPT allows users to manage their memory entries, while Anthropic's Claude retrieves past conversations only when requested [42][44]. - Gemini supports user input for memory management, enhancing its ability to remember user preferences [45]. - The M3-Agent developed by ByteDance, Zhejiang University, and Shanghai Jiao Tong University integrates long-term memory capabilities across multiple modalities, including video and audio [10][70]. Future Trends in AI Memory - The future of AI memory is expected to evolve towards multi-modal and integrated memory systems, allowing for a more comprehensive understanding of user interactions [97][106]. - There is a growing emphasis on creating memory systems that can autonomously manage and optimize their memory, akin to human cognitive processes [101][106]. - The ultimate goal is to develop AI systems that can exhibit unique personalities and emotional connections through their memory capabilities, potentially leading to the emergence of artificial general intelligence (AGI) [109][110].

Core Viewpoint - The article discusses the limitations of large language models (LLMs) regarding their context window and memory management, emphasizing the need for improved memory systems to enhance their long-term interaction capabilities [5][6][9]. Context Window Evolution - Modern LLMs typically have a limited context window, with early models like GPT-3 handling around 2,048 tokens, while newer models like Meta's Llama 4 Scout claim to manage up to 10 million tokens [2][4]. Memory Management in LLMs - LLMs face an inherent "memory defect" due to their limited context window, which hampers their ability to maintain consistency in long-term interactions [5][6]. - Recent research has focused on memory management systems like MemOS, which treat memory as a critical resource alongside computational power, allowing for continuous updates and self-evolution of LLMs [9][49]. Long Context Processing Capabilities - Long context processing capabilities are crucial for LLMs, encompassing: - Length generalization ability, which allows models to extrapolate on sequences longer than those seen during training [12]. - Efficient attention mechanisms to reduce computational and memory costs [13]. - Information retention ability, which refers to the model's capacity to utilize distant information effectively [14]. - Prompt design to maximize the advantages of long context [15]. Types of Memory in LLMs - Memory can be categorized into: - Event memory, which records past interactions and actions [18]. - Semantic memory, encompassing accessible external knowledge and understanding of the model's capabilities [19]. - Procedural memory, related to the operational structure of the system [20]. Methods to Enhance Memory and Context - Several methods to improve LLM memory and context capabilities include: - Retrieval-augmented generation (RAG), which enhances knowledge retrieval for LLMs [27][28]. - Hierarchical summarization, which recursively summarizes content to manage inputs exceeding model context length [31]. - Sliding window inference, which processes long texts in overlapping segments [32]. Memory System Design - Memory systems in LLMs are akin to databases, integrating lifecycle management and persistent representation capabilities [47][48]. - Recent advancements include the development of memory operating systems like MemOS, which utilize a layered memory architecture to manage short-term, medium-term, and long-term memory [54][52]. Innovative Memory Approaches - New memory systems such as MIRIX and Larimar draw inspiration from human memory structures, enhancing LLMs' ability to update and generalize knowledge rapidly [58][60]. - These systems aim to improve memory efficiency and model inference performance by employing flexible memory mechanisms [44].

机器之心发布 机器之心编辑部 大模型记忆管理和优化框架是当前各大厂商争相优化的热点方向，MemOS 相比现有 OpenAI 的全局记忆在大模型记忆评测集上呈现出显著的 提升，平均准确性提升超过 38.97%，Tokens 的开销进一步降低 60.95%，一举登顶记忆管理的 SOTA 框架，特别是在考验框架时序建模与检 索能力的时序推理任务上，提升比例更是达到了 159%，相当震撼！ 图 1. MemOS 项目官网报告的性能表现 在大型语言模型（LLM）一路狂飙的这几年，参数规模和算力几乎成了 AI 能力的代名词。可当大模型逐渐走进科研、产业和生活，每个人都在问一个更深 层的问题： 它究竟能不能 "记住" 点什么？ 从陪伴式对话、个性化推荐，到多轮任务协作，模型只靠一次推理、一次检索，远远不够。如何让 AI 拥有 可管理、可迁移、可共享的长期记忆 ，正在成 为新一代大模型应用的关键挑战。 近日， 记忆张量 （上海）科技有限公司联合上海交通大学、中国人民大学、同济大学、浙江大学、中国电信等多家顶尖团队发布了 MemOS（Memory Operating System） ，一套面向大模型的工业级记忆操作系统。 它的 ...