RT Avi Chawla (@_avichawla)You're in a Research Scientist interview at DeepMind.The interviewer asks:"Our investors want us to contribute to open-source.Gemini crushed benchmarks.But we'll lose competitive edge by open-sourcing it.What to do?"You: "Release a research paper."Here's what you missed:LLMs today don't just learn from raw text; they also learn from each other.For example:- Llama 4 Scout & Maverick were trained using Llama 4 Behemoth.- Gemma 2 and 3 were trained using Gemini.Distillation helps us ...

Investment Rating - The report maintains the profit forecasts, target prices, and ratings for relevant companies unchanged [6] Core Insights - The evolution of large models is fundamentally a history of combating "forgetting." The lack of a memory retention architecture leads to costly "repeated calculations" each time historical information is processed. The current model faces physical limits of memory walls and context windows. The report suggests that the AI infrastructure battlefield will increasingly focus on "model memory" starting in 2026 [3][14] - The report introduces a three-layer memory framework: short-term, medium-term, and long-term memory, each corresponding to different software and hardware requirements. This framework aims to provide a structured analysis paradigm for investment logic in AI infrastructure [14][20] Summary by Sections Short-term Memory - Short-term memory constitutes the "current view" of large models during single inference tasks. It is characterized by high-frequency read/write and extreme sensitivity to latency. The core challenge lies in the dual occupation of memory capacity and bandwidth by KV Cache. Software optimizations include PagedAttention virtualization and cutting-edge architectures like Infini-attention to support million-token context windows. Key hardware elements include HBM and on-chip SRAM [4][30][50] Medium-term Memory - Medium-term memory ensures situational continuity across sessions and is foundational for agents. The need for cross-session windows indicates a shift from stateless short-term intelligence to a complex system capable of "storage-retrieval-update-forget" dynamic management. Software advancements like GraphRAG and MemoryOS facilitate this transition, while hardware requirements include large-capacity DRAM and enterprise-grade SSDs to address high-concurrency random read/write bottlenecks [4][56] Long-term Memory - Long-term memory supports the transition from pre-training to "continuous evolution." The need for real-time updates blurs the lines between model training and inference. Long-term memory aims to break the limitations of pre-training cut-off times, allowing for continuous knowledge accumulation through implicit parameters, explicit semantics, and parameterized lookup tables. This new paradigm will drive demand for various databases and compute-storage hardware [5][21] Hardware and Software Requirements - The report outlines the hardware and software requirements for each memory layer, emphasizing the need for high-bandwidth memory (HBM), large-capacity DRAM, and enterprise SSDs. It also highlights the importance of software solutions like KV Cache management and advanced attention mechanisms to optimize memory usage and enhance performance [16][50][64]

Investment Rating - The report maintains an "Overweight" rating for the technology and computer sectors [7]. Core Insights - The AI industry is entering a new paradigm characterized by the Scaling Law 2.0, where synthetic data expands the training data ceiling, and the Mid Training paradigm reshapes model evolution paths [2][3]. - The commercial application of AI is transitioning into a scaling phase, with the integration of agent capabilities and transaction loops accelerating industry implementation [2][6]. Summary by Sections Models - Computing power expansion remains the core growth engine, with representative model training computing power expected to grow at an annual rate of 4-5 times from 2010 to 2024, with leading models achieving up to 9 times [3][13]. - The cost of complete training for frontier models is projected to reach the billion-dollar level by 2027 [3][13]. Training - The Mid Training paradigm expands training boundaries by integrating reinforcement learning (RL) into the middle stage, enhancing data generation and optimal allocation [4][16]. - This approach significantly increases data utilization efficiency and is expected to break traditional performance limits [4][16]. Agents - GPT-5 establishes a "unified system" direction, promoting standardization in agent architecture through adaptive collaboration between fast and deep thinking [5][19]. - The real-time router dynamically allocates computing resources based on task complexity, enhancing response efficiency and stability in complex scenarios [5][19]. Applications - The integration of agent capabilities into commercial transactions marks a new phase of AI applications, with OpenAI's Agentic Commerce Protocol enabling AI agents to execute purchases directly [6][22]. - The global AI application landscape is evolving through three stages: productization in 2023, commercialization trials in 2024, and scaling implementation in 2025 [25][26]. - Domestic AI applications are accelerating, with significant advancements in commercial capabilities following the release of models like DeepSeek-R1 [26].

You're in a Research Scientist interview at OpenAI.The interviewer asks:"Our investors want us to contribute to open-source.o3 crushed benchmarks.But we can lose a competitive edge by open-sourcing it.What do we do?"You: "Release the research paper."Interview over.You forgot that LLMs don't just learn from raw text; they also learn from each other.For example:- Llama 4 Scout & Maverick were trained using Llama 4 Behemoth.- Gemma 2 and 3 were trained using Gemini.Distillation helps us do so, and the visual e ...

Core Viewpoint - The article discusses the limitations of large language models (LLMs) regarding their context window and memory management, emphasizing the need for improved memory systems to enhance their long-term interaction capabilities [5][6][9]. Context Window Evolution - Modern LLMs typically have a limited context window, with early models like GPT-3 handling around 2,048 tokens, while newer models like Meta's Llama 4 Scout claim to manage up to 10 million tokens [2][4]. Memory Management in LLMs - LLMs face an inherent "memory defect" due to their limited context window, which hampers their ability to maintain consistency in long-term interactions [5][6]. - Recent research has focused on memory management systems like MemOS, which treat memory as a critical resource alongside computational power, allowing for continuous updates and self-evolution of LLMs [9][49]. Long Context Processing Capabilities - Long context processing capabilities are crucial for LLMs, encompassing: - Length generalization ability, which allows models to extrapolate on sequences longer than those seen during training [12]. - Efficient attention mechanisms to reduce computational and memory costs [13]. - Information retention ability, which refers to the model's capacity to utilize distant information effectively [14]. - Prompt design to maximize the advantages of long context [15]. Types of Memory in LLMs - Memory can be categorized into: - Event memory, which records past interactions and actions [18]. - Semantic memory, encompassing accessible external knowledge and understanding of the model's capabilities [19]. - Procedural memory, related to the operational structure of the system [20]. Methods to Enhance Memory and Context - Several methods to improve LLM memory and context capabilities include: - Retrieval-augmented generation (RAG), which enhances knowledge retrieval for LLMs [27][28]. - Hierarchical summarization, which recursively summarizes content to manage inputs exceeding model context length [31]. - Sliding window inference, which processes long texts in overlapping segments [32]. Memory System Design - Memory systems in LLMs are akin to databases, integrating lifecycle management and persistent representation capabilities [47][48]. - Recent advancements include the development of memory operating systems like MemOS, which utilize a layered memory architecture to manage short-term, medium-term, and long-term memory [54][52]. Innovative Memory Approaches - New memory systems such as MIRIX and Larimar draw inspiration from human memory structures, enhancing LLMs' ability to update and generalize knowledge rapidly [58][60]. - These systems aim to improve memory efficiency and model inference performance by employing flexible memory mechanisms [44].

Core Viewpoint - Meta's CEO Mark Zuckerberg is under pressure to enhance the company's AI capabilities and is adopting a more hands-on approach to management, including the establishment of a Corporate Venture Capital (CVC) unit to attract top talent and improve performance in the AI sector [2][8]. Group 1: Meta's Current Challenges - Zuckerberg's recent management style has shifted to a more direct and micro-level approach, reallocating resources to the GenAI team to boost the performance of LLaMA [2][4]. - There is a growing concern about talent retention at Meta, with reports of AI engineers leaving for competitors like OpenAI and Anthropic, often with offers exceeding $2 million [6][7]. - The AI landscape is becoming increasingly competitive, with Meta's LLaMA struggling to keep pace with rivals like Qwen and DeepSeek, leading to a perception of stagnation in Meta's AI initiatives [6][12]. Group 2: Establishment of CVC - Historically, Meta has not had a dedicated CVC, relying instead on its corporate development teams for acquisitions [4][5]. - The decision to form a CVC is part of Zuckerberg's broader strategy to create a "superintelligence unit" aimed at revitalizing Meta's AI efforts [8][10]. - Meta's investment in the venture fund NFDG, led by Daniel Gross, is a strategic move to gain access to top talent and innovative projects in the AI space [9][12]. Group 3: Financial Implications and Market Dynamics - The AI investment landscape is currently dominated by corporate investments, which accounted for approximately 75% of the total funding in 2023, indicating a scarcity of available high-quality targets [12][13]. - Meta's recent acquisition of Scale AI for $14.8 billion is seen as a critical step in its strategy to bolster its AI capabilities [7][12]. - The overall number of AI startups has decreased significantly, with a reported 81% drop in new AI companies since the peak in 2021, complicating Meta's efforts to secure talent and technology [12][13].

Core Insights - Meta's CEO Mark Zuckerberg is experiencing frustration as the company struggles to keep pace with competitors in the AI space, particularly in light of its underwhelming performance in the metaverse and AR/VR sectors [1][2] - Despite Meta's strong financial performance and stock price nearing historical highs, there is growing anxiety about the company's future direction and competitiveness in AI [1][2] Group 1: Management Changes and Strategies - Zuckerberg has taken a hands-on approach to AI management, reallocating resources from foundational AI research to the GenAI team to enhance the performance of LLaMA [2] - The restructuring includes demoting the head of the GenAI team and splitting it into two groups, reflecting Zuckerberg's intense pressure to deliver results [2] - Meta's lack of a dedicated Corporate Venture Capital (CVC) team has prompted Zuckerberg to consider establishing one to better compete in the AI landscape [4][7] Group 2: Talent Acquisition Challenges - Meta is facing significant talent retention issues, with reports of AI engineers leaving for competitors like OpenAI and Anthropic, often with offers exceeding $2 million [6] - Zuckerberg's ambitious "superintelligence unit" plan aims to recruit top industry talent, offering salaries that could reach nine figures [6][7] - The difficulty in attracting talent is compounded by the competitive landscape, where even substantial financial incentives have not been enough to secure top candidates [10][12] Group 3: Investment and Acquisition Strategies - Meta's acquisition of Scale AI for $14.8 billion is part of a broader strategy to bolster its AI capabilities and leadership [6][12] - The company is also investing in Daniel Gross's venture fund, NFDG, to gain access to top talent and expertise in AI [7][8] - The overall investment landscape in AI is becoming increasingly competitive, with a significant drop in the number of new AI startups and rising costs for quality acquisitions [11][12]

Core Viewpoint - Multiverse Computing has successfully raised €189 million (approximately $215 million) in a Series B funding round, leveraging its technology "CompactifAI" to compress large language models (LLMs) significantly while maintaining performance [1][2]. Group 1: Funding and Investment - The Series B funding round was led by Bullhound Capital, with participation from various investors including HP Tech Ventures, SETT, Forgepoint Capital International, CDP Venture Capital, Santander Climate VC, Toshiba, and the Basque venture capital group [1]. - To date, the company has raised approximately $250 million in total funding [2]. Group 2: Technology and Product Offering - CompactifAI is a compression technology inspired by quantum computing, capable of reducing the size of LLMs by up to 95% without compromising model performance [2]. - Multiverse offers compressed versions of well-known open-source LLMs, including Llama 4 Scout, Llama 3.3 70B, Llama 3.1 8B, and Mistral Small 3.1, with plans to release more models soon [2]. - The company claims that its models operate 4 to 12 times faster than uncompressed versions, with inference costs reduced by 50% to 80% [3]. Group 3: Market Applications and Accessibility - Some of Multiverse's models are designed to be compact and energy-efficient, enabling them to run on personal computers, smartphones, cars, drones, and even Raspberry Pi devices [3]. - The Llama 4 Scout Slim version costs $0.10 per million tokens on AWS, compared to $0.14 for the original version, showcasing significant cost savings [3]. Group 4: Leadership and Expertise - The company is backed by strong technical expertise, with co-founder and CTO Román Orús known for his pioneering research in tensor networks, which are tools for simulating quantum computers on conventional machines [4]. - Co-founder and CEO Enrique Lizaso Olmos has a background in mathematics and extensive experience in the banking sector, previously serving as the Deputy CEO of Unnim Banc [4].

Group 1 - Meta is establishing a new lab focused on "Superintelligence" to develop AI systems that surpass human intelligence in reasoning, problem-solving, creativity, and decision-making [1][3] - Meta has agreed to acquire 49% of Scale AI for $14.8 billion, which is approximately 106.14 billion RMB [1][3] - Alexander Wang, the 28-year-old CEO of Scale AI, is invited to join Meta's new lab, highlighting Meta's strategy to attract top talent in the AI field [1][4] Group 2 - Meta is offering compensation packages ranging from seven to nine figures to recruit top researchers from companies like OpenAI and Google, with some already agreeing to join [4][9] - Scale AI, founded in 2016, provides data labeling solutions and reported a revenue of $870 million in the previous year, with expectations to double to over $2 billion this year [3][9] - Meta's AI efforts are led by two groups: a generative AI team and a fundamental AI research lab, with Yann LeCun, a Turing Award winner, overseeing the latter [4][9] Group 3 - Meta's recent AI model testing faced criticism, with external researchers questioning the objectivity of its benchmark tests [5][8] - The company aims to regain its competitive edge in AI, especially after the rise of ChatGPT, which has intensified competition in the tech industry [9][10] - Meta's previous focus on open-source large models and social platform AI tools has led to a fragmented strategy, prompting the need for a more cohesive approach [10]

Core Insights - Meta Platforms is delaying the release of its "Behemoth" AI model due to concerns about its capabilities and the significance of improvements over earlier versions [1][3] - The initial release was scheduled for April to align with Meta's first AI conference but has now been postponed to fall or later [2][3] Development Timeline - Behemoth was originally set for an April release, which was later pushed to June, and is now delayed further [2][3] - The company had previously described Behemoth as "one of the smartest LLMs in the world" and its most powerful model to date [3][5] Recent Developments - In April, Meta released the latest versions of its LLM, Llama 4 Scout and Llama 4 Maverick, while previewing Behemoth [5]