Chan Ye Xin Xi Wang·2026-01-01 03:22Core Insights - The hybrid expert model (MoE) is recognized as a "structural revolution" in artificial intelligence, enabling the construction of ultra-large-scale and high-efficiency models through its sparse activation design [1][7] - The market size for China's MoE industry is projected to reach approximately 148 million yuan in 2024, reflecting a year-on-year growth of 43.69% [1][7] - The sparse activation mechanism allows models to scale to trillions of parameters at a significantly lower computational cost compared to traditional dense models, achieving a revolutionary balance between performance, efficiency, and cost [1][7] Industry Overview - MoE is a neural network architecture that enhances performance and efficiency by dynamically integrating multiple specialized sub-models (experts), focusing on a "divide-and-conquer strategy + conditional computation" [2][3] - The core characteristics of MoE include high parameter capacity and low computational cost, activating only a small portion of total parameters to expand model size [2][3] - MoE faces technical challenges such as load balancing, communication overhead among experts, and high memory requirements, while offering advantages like task specificity, flexibility, and efficiency [2][3] Industry Development History - The MoE concept originated from the "adaptive mixture of local experts" theory proposed by Michael Jordan and Geoffrey Hinton in 1991, focusing on efficient collaboration through a gating network [3][4] - Significant advancements occurred in 2017 when Google introduced sparse gating mechanisms in LSTM networks, leading to substantial reductions in computational costs and performance breakthroughs in NLP tasks [3][4] - The MoE technology has rapidly evolved alongside deep learning and big data trends, with notable models like Mistral AI's Mixtral 8x7B and DeepSeek-MoE series pushing the boundaries of performance and efficiency [3][4] Industry Value Chain - The upstream of the MoE industry includes chips, storage media, network devices, and software tools for instruction sets and communication libraries [6] - The midstream focuses on the development and optimization of MoE models, while the downstream applications span natural language processing, computer vision, multimodal large models, and embodied intelligence [6] - The natural language processing market in China is expected to reach approximately 12.6 billion yuan in 2024, growing by 14.55% year-on-year, driven by technological breakthroughs and increasing demand across various sectors [6] Market Size - The MoE industry in China is projected to reach a market size of about 148 million yuan in 2024, with a year-on-year growth rate of 43.69% [1][7] - The technology's advantages are attracting significant investments from research institutions, large tech companies, and AI startups, facilitating the transition from technical prototypes to scalable commercial applications [1][7] Key Company Performance - The MoE industry in China is characterized by a competitive landscape involving "open-source pioneers, large enterprises, and vertical deep-divers," with market concentration undergoing dynamic reshaping [8][9] - Leading companies like Kunlun Wanwei and Tencent are leveraging technological innovation and product advantages to establish a strong market position [8][9] - Kunlun Wanwei launched the first domestic open-source model based on MoE architecture in February 2024, achieving a threefold increase in inference efficiency compared to dense models [9] Industry Development Trends - The demand for multimodal data is driving the integration of MoE architecture with technologies like computer vision and speech recognition, making multimodal MoE models mainstream [10] - Breakthroughs in sparse activation and expert load balancing technologies are enhancing the stability and inference efficiency of large-scale MoE models [11] - The construction of ecosystems around open-source frameworks and domestic computing power is accelerating the large-scale implementation of MoE in various fields [12]