芯片初创公司，攻破内存墙

Core Viewpoint - The article discusses the significant demand for memory bandwidth and capacity in AI inference workloads, highlighting d-Matrix's innovative 3D Stacked In-Memory Compute (3DIMC) architecture as a solution to address these challenges [2][5][8]. Group 1: Company Overview - d-Matrix was founded in 2019 by Sid Sheth and Sudeep Bhoja, both former executives at Inphi Corp, which was acquired by Marvell for $10 billion in 2020 [2]. - The company aims to develop memory compute chip-level technology that offers greater bandwidth than traditional DRAM at a lower cost compared to High Bandwidth Memory (HBM) [2]. Group 2: Technology and Innovation - The 3DIMC architecture integrates 3D stacked memory with computing capabilities, significantly reducing latency and enhancing bandwidth while improving efficiency [3][8]. - d-Matrix's technology utilizes LPDDR5 memory and connects Digital In-Memory Compute (DIMC) hardware to memory via an intermediary layer, optimizing for matrix-vector multiplication, a key operation in transformer-based models [3][5]. Group 3: Performance Expectations - d-Matrix anticipates that 3DIMC will enhance memory bandwidth and capacity for AI inference workloads by several orders of magnitude, enabling efficient and cost-effective large-scale operations as new models and applications emerge [5][9]. - The next-generation architecture, Raptor, is expected to incorporate 3DIMC, aiming for a tenfold increase in memory bandwidth and energy efficiency compared to HBM4 when running AI inference workloads [5][9]. Group 4: Market Trends and Predictions - The article notes a significant shift from AI training to AI inference, with d-Matrix positioned to meet the growing demand for faster and larger memory solutions driven by large language models (LLMs) [6][7]. - Sheth predicts that the reliance on transformer models will dominate AI computing for the next 5 to 10 years, leading to a surge in AI inference workloads [6].