Seek .(US:SKLTY) Sou Hu Cai Jing·2026-01-07 09:13Core Insights - DeepSeek has introduced a new architecture called mHC aimed at addressing stability issues in large-scale model training while maintaining performance improvements [1][11]. Group 1: Problem Identification - Large models face a dilemma in training stability, where traditional single-channel connections lead to information congestion as model size increases [3][5]. - Previous solutions, like the hyper-connection approach, improved efficiency but introduced new issues such as uncontrolled information amplification or suppression, leading to gradient explosion and training failures [5][7][9]. Group 2: mHC Architecture - The mHC architecture incorporates an intelligent scheduling system for multi-channel connections, utilizing the Sinkhorn-Knopp algorithm to maintain energy conservation during information transmission [11][13]. - Additional design features include non-negative constraints on input-output mappings to prevent useful signal loss due to coefficient cancellation [15]. Group 3: Infrastructure Optimization - DeepSeek has optimized its infrastructure by merging multiple computation steps into a single operator, reducing memory read/write cycles and employing recomputation strategies to lower memory usage [16][18]. - These optimizations have resulted in significant stability improvements with minimal increases in training time, even at an expansion factor of 4 [18]. Group 4: Performance Validation - Testing on various model sizes, particularly a 27 billion parameter model, demonstrated that mHC effectively resolved training instability issues, achieving lower loss values compared to traditional baseline models [21][22]. - The performance advantages of mHC were consistent across different model sizes, indicating its practical value for both small and large models [24]. Group 5: Industry Implications - The introduction of mHC suggests a shift in the industry towards refined architectural designs rather than merely increasing parameters and computational power, potentially lowering entry barriers for smaller companies in the large-scale model domain [26][29]. - This pragmatic technological innovation is expected to facilitate the deployment of AI technologies, making it easier for more enterprises to engage in large-scale model development [29].