DeepSeek上新mHC，R2还远吗？

Core Insights - DeepSeek has introduced a new neural network architecture optimization called mHC (Manifold-Constrained Hyper-Connections), which is expected to significantly impact the AI industry, including large models and chips [1][5][9] Group 1: mHC Architecture - The mHC architecture builds on the Hyper-Connections (HC) framework released by the Byte Bean team in November 2024, aiming to replace the nearly decade-old ResNet architecture [5] - mHC introduces a Manifold-Constrained approach using the Sinkhorn-Knopp algorithm to stabilize signal propagation during training, addressing issues of signal explosion and instability in large model training [5][6] - In training demonstrations with 27 billion parameters, mHC maintained a signal amplification of only 1.6 times, while HC experienced a catastrophic failure with a 3000 times amplification [6][8] Group 2: Performance and Efficiency - mHC shows a significant reduction in training loss and improved performance on challenging tasks, with over 2% enhancement in reasoning and reading comprehension benchmarks compared to traditional architectures [6][8] - The additional training time overhead for mHC, even with a fourfold expansion of residual channels, is only 6.7%, indicating a focus on cost-effectiveness and efficiency [8] Group 3: Industry Impact and Reactions - The release of mHC has sparked high discussion levels among researchers and industry professionals, with expectations of a paradigm shift in large model architectures by 2026 [9][10] - Competitors are already responding, with new architectures like Deep Delta Learning emerging shortly after mHC's announcement, indicating a potential chain reaction in AI architecture development [9][10] - Analysts predict that DeepSeek may make significant announcements around the Lunar New Year, potentially unveiling the long-awaited R2 model or a faster universal model V4 [10] Group 4: Compatibility and Market Dynamics - mHC's architecture is primarily designed for NVIDIA's supernode links, raising concerns about compatibility with domestic chips, which may require enhanced adaptation efforts [11] - As U.S. AI chip manufacturers gradually exit the Chinese market due to geopolitical factors, domestic chipmakers are accelerating their development and ecosystem building to adapt to DeepSeek's models [12]