01 不要拥有，要体验 文章的第一个问题：假如人生每过十年财产清零，你现在最应该做什么？ 逢十生日那一天，财产定期清零，这个问题虽然科幻，但并不难回答，丧失了"钱生钱"的复利效应，财富的积累就失去了意义。 钱仍然有意义，只是存款没有意义，投资也没有意义，任何不在十年内被花掉的钱，都会失去意义，类似买那种可以用一辈子的房子，就成为毫无意义的 浪费。 当每一个人都不再为"以后"而过度储蓄和吝啬，更重要的事就变成，如何花钱才有意义？ 活在当下，不再是一句鸡汤，而是生存的必然选择：体验消费比实物消费更重要，吃饭泡吧比买名牌时装重要，旅游比豪车重要。 时间成为最珍贵的东西，加班给十倍工资都没人肯干，让我们充分享受这十年中的每一天吧。 还会有人认真地面对个人爱好，真诚地学习乐器、绘画、烹饪、运动等，而不是当成谋生的手段。 不过，这样的世界仍然有贫富差别，每个人赚钱的能力不同，所以很多人还会为下一个十年做准备，除了消费，最重要的就是积累知识与培养能力，这是 规则中允许你带到下一个十年的。 而且，在一个没有富二代，没有老钱的时代，唯一能给你带来财富的只有努力与知识，这是一个真正的"知识比财富更重要"的时代。 为了拥抱周期性的 ...

来自OpenAI、谷歌等公司的小部分但日益增长的AI开发者群体认为，当前的技术路径无法实现生物 学、医学等领域的重大突破，也难以避免简单错误。这一观点正在引发行业对数十亿美元投资方向的质 疑。 据The Information周二报道，上周在圣地亚哥举行的神经信息处理系统大会（NeurIPS）上，众多研究 人员讨论了这一话题。他们认为，开发者必须创造出能在部署后持续获取新能力的AI，这种"持续学 习"能力类似人类的学习方式，但目前尚未在AI领域实现。 然而，技术局限已拖慢企业客户对AI代理等新产品的采购。模型在简单问题上持续犯错，AI代理在缺 乏AI提供商大量工作确保其正确运行的情况下往往表现不佳。 这些质疑声与部分AI领袖的乐观预测形成对比。Anthropic首席执行官Dario Amodei上周表示，扩展现有 训练技术就能实现通用人工智能（AGI），OpenAI首席执行官Sam Altman则认为两年多后AI将能自我 改进。但如果质疑者是对的，这可能令OpenAI和Anthropic明年在强化学习等技术上投入的数十亿美元 面临风险。 尽管存在技术局限，当前AI在写作、设计、购物和数据分析等任务上的表现仍推 ...

About Us 我们是一个对 AI 和 foundation model 痴迷的团队。 2022 年秋，我们在硅谷看到了 AI 的火苗，从此只专注研究 AI。 专注研究和投资 AI，让我们取得了还不错的成绩。我们在管 AUM 超过 15 亿美金，有 5 亿美元在投的长 线基金，一二级联动，有足够的子弹抓住 AI 机会。 我们过去投资并见证了 6 家 portfolio 从数十亿，数 百亿美金，成长为千亿美金公司——这也是拾象的寓意，只研究全球最重要的技术变化，投资有大象级 潜力的公司。 我们有数位千亿美金业务的 CEO / leadership 提供洞察，帮 portfolio 做好 AI 和全球化。 我们通过海外独角兽和 AI 讨论社群，持续讨论重要问题，帮助和影响了中美两地的华人创业者，也在 AI 从业者中获取了一些宝贵的信任。 现在，我们希望邀请你加入，一起做 全球 AI 投资，一起捕捉大机会，成为 AI 领域的最佳投手 。我们是 一个年轻（平均年龄不到 30 岁）、扁平、高人才密度的团队，推崇 high-trust，low-ego，团队内信息极 度透明，讨论氛围热烈。 我们尤其喜欢的特质是：对 AI ...

Core Insights - Tsinghua University's research team has proposed a novel continual learning (CL) framework called H-embedding guided hypernetwork, which addresses the issue of catastrophic forgetting in AI models by focusing on task relationships [1][4][21] - The framework aims to enhance the model's ability to absorb new knowledge while maintaining performance on old tasks, thus facilitating long-term intelligence in AI systems [1][21] Group 1: Problem Identification - Catastrophic forgetting is a significant bottleneck in the practical application of continual learning, where models forget old knowledge when learning new tasks [1][4] - Existing CL methods primarily adopt a model-centric approach, neglecting the intrinsic relationships between tasks, which directly influence knowledge transfer efficiency [1][8] Group 2: Proposed Solution - The H-embedding guided hypernetwork framework introduces a task-relation-centric approach, constructing transferable task embeddings (H-embedding) before learning new tasks [4][6] - This method allows for explicit encoding of task relationships in the CL process, enabling the model to manage knowledge transfer more effectively [6][21] Group 3: Methodology - H-embedding is derived from H-score, which quantifies the transfer value from old tasks to current tasks, facilitating efficient computation of transferability [9][11] - The framework employs a hypernetwork to generate task-specific parameters based on the H-embedding, allowing for automatic adjustment of parameters according to task differences [12][17] Group 4: Experimental Results - The proposed framework has shown superior performance across multiple CL benchmarks, including CIFAR-100, ImageNet-R, and DomainNet, demonstrating its robustness and scalability [18][20] - The model exhibits strong forward and backward transfer capabilities, with minimal interference from new tasks on old tasks, and effectively absorbs knowledge from previous tasks [20] Group 5: Future Directions - The research indicates potential applications of task-structure-aware methods in cross-modal incremental learning, long-term task adaptation for large models, and automated learning sequence planning [21][23] - This approach aims to contribute to the development of more scalable and adaptable general AI systems [21]

Core Insights - The global AI spending is projected to approach $1.5 trillion by 2025 and exceed $2 trillion by 2026, with Nvidia's CEO estimating that infrastructure investments in AI could reach $3 to $4 trillion this decade, marking a new industrial revolution [1][34] - The AI industry is transitioning from an era focused on scaling resources to one centered on research and innovation, as highlighted by Ilya Sutskever, the former chief scientist of OpenAI [2][5][6] Group 1: Transition in AI Development - The era of simply scaling parameters, compute power, and data is coming to an end, as the industry consensus has led to a resource arms race rather than true innovation [7][9] - Sutskever emphasizes that the future of AI will depend on new training methods rather than just increasing GPU counts, indicating a shift in competitive advantage [7][12] Group 2: Limitations of Current Models - Current large models exhibit high benchmark scores but often fail to deliver real economic value, revealing a disconnect between perceived capability and practical application [9][10] - The models are criticized for their lack of generalization ability, often performing well in tests but struggling with real-world tasks due to systemic flaws in their training processes [11][16] Group 3: Need for New Training Approaches - Sutskever argues that existing training methods, including pre-training and reinforcement learning, have fundamental limitations that prevent models from truly understanding and applying knowledge [18][20] - The focus should shift towards continuous learning and self-evaluation, allowing models to adapt and improve in real-world scenarios rather than being static after initial training [27][29] Group 4: Safety and Alignment in AI - The concept of safety in AI should be integrated from the training phase, as the ability to generalize and understand context is crucial for reliable performance in unknown situations [25][26] - Sutskever's new approach advocates for a model that can learn continuously and align with human values, moving away from a one-time training paradigm [28][30] Group 5: Implications for the Future of AI - The shift from resource-based competition to method-based innovation is seen as a critical turning point in the AI industry, with research capabilities becoming the key differentiator [33] - The current evaluation systems are evolving, as the focus on merely increasing model size and parameters is proving insufficient for addressing the complexities of AI deployment [33]

Group 1 - The article discusses the concept of "Nested Learning" proposed by Google, which aims to address the memory management issues in LLMs (Large Language Models) and the challenges of catastrophic forgetting [5][6][8] - Nested Learning is presented as a multi-layered optimization problem, where models are seen as a series of interconnected sub-problems, allowing for the simultaneous learning of new skills while avoiding the loss of previously acquired knowledge [6][7] - The research introduces the "Continuous Memory System" (CMS), which treats memory as a system of multiple modules that update at different frequencies, enhancing the model's ability to manage memory effectively [6][7] Group 2 - The article highlights the importance of improving LLMs' memory capabilities to enable continual learning, allowing AI to retain contextual experiences, semantic knowledge, and procedural skills [8] - A proposed three-layer memory architecture includes Model Weights for general knowledge, KV Cache for intermediate results, and Context for relevant background information, facilitating appropriate responses from the model [8]

Core Insights - Google has introduced a new machine learning paradigm called Nested Learning, which allows models to continuously learn new skills without forgetting old ones, marking a significant advancement towards AI that evolves like the human brain [1][3][4]. Group 1: Nested Learning Concept - Nested Learning treats machine learning models as a series of interconnected optimization sub-problems, enabling a more efficient learning system [6][11]. - The approach bridges the gap between model architecture and optimization algorithms, suggesting they are fundamentally the same and can be organized into hierarchical optimization systems [7][16]. - This paradigm allows for different components of a model to update at varying frequencies, enhancing the model's ability to manage long-term and short-term memory [15][20]. Group 2: Implementation and Architecture - Google has developed a self-modifying architecture called Hope, based on Nested Learning principles, which outperforms existing models in language modeling and long-context memory management [8][24]. - Hope is an evolution of the Titans architecture, designed to execute infinite levels of contextual learning and optimize its memory through a self-referential process [24][26]. Group 3: Experimental Results - Evaluations show that Hope exhibits lower perplexity and higher accuracy in various language modeling and common-sense reasoning tasks compared to other architectures [27][30]. - The performance of different architectures, including Hope, Titans, and others, was compared in long-context tasks, demonstrating the effectiveness of the Nested Learning framework [30]. Group 4: Future Implications - Nested Learning provides a theoretical and practical foundation for bridging the gap between current LLMs' limitations and the superior continuous learning capabilities of the human brain, paving the way for the development of self-improving AI [30].

Core Insights - The article discusses the recent advancements in large language models (LLMs) regarding their ability to achieve continual learning and self-evolution, addressing criticisms about their lack of genuine learning capabilities [1][2]. Group 1: Paths to Continual Learning - The ability of LLMs to learn continuously is fundamentally linked to their memory depth and plasticity, with three main paths identified for enhancing this capability [2]. - The first path involves modifying the "context" or "working memory" of the model through In-Context Learning (ICL), where new information is provided in prompts to help the model learn to solve specific problems [4][6]. - The second path introduces an "external memory bank" (RAG), allowing models to access and maintain an external database for comparison and retrieval, exemplified by Google's DeepMind's "Reasoningbank" [7]. - The third path focuses on parameter-level continual learning, which has faced challenges due to the complexities and instabilities associated with methods like Reinforcement Learning (RL) and Low-Rank Adaptation (LoRA) [10][11]. Group 2: Sparse Memory Fine-Tuning - Meta AI's recent paper introduces Sparse Memory Fine-Tuning (SFT) as a solution to the challenges of traditional SFT, particularly addressing the issue of catastrophic forgetting [11][28]. - The proposed method involves a three-step process: modifying the architecture to include a memory layer, using TF-IDF to identify which parameters to update, and performing sparse updates to only the most relevant parameters [12][22][23]. - This new approach has shown significant improvements, with models experiencing only an 11% drop in performance on original tasks after learning new facts, compared to 71% and 89% drops with LoRA and full fine-tuning, respectively [23][25]. Group 3: Implications for the Future of LLMs - The advancements in SFT suggest a potential shift in how models can be updated safely and effectively, moving away from static tools to dynamic agents capable of continuous learning [31][32]. - The successful implementation of these methods could mark the beginning of a new era for self-evolving models, aligning with the vision of models that grow and adapt through experience [31][32].

Core Insights - A significant 96.0% of surveyed young professionals believe that personal growth should be prioritized after entering the workforce, emphasizing the importance of continuous learning for career advancement [1][2][5] Group 1: Importance of Continuous Learning - Continuous learning is viewed as essential for career development, with professionals acknowledging that the knowledge gained post-graduation is crucial for determining future career paths [2][4] - 54.8% of respondents feel that ongoing self-learning allows them to perform more confidently at work, while 47.1% report increased self-confidence and a sense of achievement [5] Group 2: Areas of Focus for Growth - The survey indicates that 70.9% of young professionals prioritize enhancing their professional skills, followed by 68.0% focusing on work-related tasks, and 53.4% on interpersonal communication [3][5] - Other areas of interest include financial literacy (41.7%), time management (41.1%), and personal development (39.9%) [3] Group 3: Personal Experiences and Outcomes - Professionals report that engaging in continuous learning has led to a more fulfilling daily routine and increased self-confidence, with many feeling more equipped to handle workplace challenges [4][5] - The pursuit of personal interests, such as hobbies and skills outside of work, is also seen as beneficial for overall well-being and career satisfaction [4]

Core Insights - The article discusses the current state and future of AI, particularly focusing on the limitations of reinforcement learning and the timeline for achieving Artificial General Intelligence (AGI) [5][6][10]. Group 1: AGI and AI Development - AGI is expected to take about ten years to develop, contrary to the belief that this year would be the year of agents [12][13]. - Current AI agents, such as Claude and Codex, are impressive but still lack essential capabilities, including multi-modal abilities and continuous learning [13][14]. - The industry has been overly optimistic about the pace of AI development, leading to inflated expectations [12][15]. Group 2: Limitations of Reinforcement Learning - Reinforcement learning is criticized as being inadequate for replicating human learning processes, as it often relies on trial and error without a deep understanding of the problem [50][51]. - The approach of reinforcement learning can lead to noise in the learning process, as it weights every action based on the final outcome rather than the quality of the steps taken [51][52]. - Human learning involves a more complex reflection on successes and failures, which current AI models do not replicate [52][53]. Group 3: Future of AI and Learning Mechanisms - The future of AI may involve more sophisticated attention mechanisms and learning algorithms that better mimic human cognitive processes [33][32]. - There is a need for AI models to develop mechanisms for long-term memory and knowledge retention, which are currently lacking [31][32]. - The integration of AI into programming and development processes is seen as a continuous evolution rather than a sudden leap to superintelligence [45][47].