Core Viewpoint - The article discusses the architecture and data flow of Huawei's Cloud Matrix 384, emphasizing the integration of optical and electrical interconnections in its network design [2][3][9]. Group 1: Data Transmission Layers - The Cloud Matrix 384 includes three main data transmission layers: UB Plane, RDMA Plane, and VPC Plane, each serving distinct roles in data processing and communication [5][7]. - The UB Plane connects all NPU and CPU with a non-blocking full-mesh topology, providing a unidirectional bandwidth of 392GB/s per Ascend 910C [7]. - The RDMA Plane facilitates horizontal scaling communication between supernodes using RoCE protocol, primarily connecting NPUs for high-speed KV Cache transfer [7]. - The VPC Plane connects supernodes to broader data center networks, managing tasks such as storage access and external service communication [7]. Group 2: Optical and Electrical Interconnections - Although the Cloud Matrix 384 is often referred to as a purely optical interconnection system, it also utilizes electrical interconnections for short distances to reduce costs and power consumption [9]. - The article highlights the necessity of both optical and electrical connections in achieving efficient data flow within the system [9]. Group 3: Scale-Up and Scale-Out Calculations - For Scale-Up, each server's UB Switch chip corresponds to a bandwidth of 448GBps, requiring 56 400G optical modules or 28 800G dual-channel optical modules per server [12]. - The ratio of NPUs to 400G optical modules in Scale-Up is 1:14, and to 800G modules is 1:7 [12]. - For Scale-Out, a Cloud Matrix node consists of 12 Compute cabinets, and the optical module demand ratio is approximately 1:4 for NPUs to 400G optical modules [14].

Core Viewpoint - The article discusses the competition between InfiniBand and Ethernet in AI networking, highlighting the advantages of Ethernet in terms of cost, scalability, and compatibility with existing infrastructure [6][8][22]. Group 1: AI Networking Overview - AI networks are primarily based on InfiniBand due to NVIDIA's dominance in the AI server market, but Ethernet is gaining traction due to its cost-effectiveness and established deployment in large-scale data centers [8][20]. - The establishment of the "Ultra Ethernet Consortium" (UEC) aims to enhance Ethernet's capabilities for high-performance computing and AI, directly competing with InfiniBand [8][9]. Group 2: Deployment Considerations - Teams face four key questions when deploying AI networks: whether to use existing TCP/IP networks or build dedicated high-performance networks, whether to choose InfiniBand or Ethernet-based RoCE, how to manage and maintain the network, and whether it can support multi-tenant isolation [9][10]. - The increasing size of AI models, often reaching hundreds of billions of parameters, necessitates distributed training, which relies heavily on network performance for communication efficiency [10][20]. Group 3: Technical Comparison - InfiniBand offers advantages in bandwidth and latency, with capabilities such as high-speed data transfer and low end-to-end communication delays, making it suitable for high-performance computing [20][21]. - Ethernet, particularly RoCE v2, provides flexibility and cost advantages, allowing for the integration of traditional Ethernet services while supporting high-performance RDMA [18][22]. Group 4: Future Trends - In AI inference scenarios, Ethernet is expected to demonstrate greater applicability and advantages due to its compatibility with existing infrastructure and cost-effectiveness, leading to more high-performance clusters being deployed on Ethernet [22][23].

Core Insights - The advancement of artificial intelligence (AI) relies on the exponential growth of AI supercomputers, with training compute power increasing by 4.1 times annually since 2010, leading to breakthroughs in various AI applications [1][13] - The performance of leading AI supercomputers doubles approximately every nine months, driven by a 1.6 times annual increase in the number of chips and their performance [2][3] - By 2025, the most powerful AI supercomputer, xAI's Colossus, is estimated to have a hardware cost of $7 billion and a power demand of around 300 megawatts, equivalent to the electricity consumption of 250,000 households [3][41] Group 1: AI Supercomputer Performance and Growth - The performance of leading AI supercomputers is projected to grow at an annual rate of 2.5 times, with private sector systems growing even faster at 3.1 times [21][29] - The number of AI chips in top supercomputers is expected to increase from over 10,000 in 2019 to over 200,000 by 2024, exemplified by xAI's Colossus [2][24] - The energy efficiency of AI supercomputers is improving, with a yearly increase of 1.34 times, primarily due to the adoption of more energy-efficient chips [45][49] Group 2: Hardware Costs and Power Demand - The hardware costs of leading AI supercomputers are projected to double annually, reaching approximately $2 billion by 2030 [50][73] - Power demand for these supercomputers is expected to grow at a rate of 2.0 times per year, potentially reaching 9 gigawatts by 2030, which poses significant challenges for infrastructure [41][75] - The rapid increase in power demand may lead companies to adopt distributed training methods to manage workloads across multiple locations [76][77] Group 3: Market Dynamics and Geopolitical Implications - The private sector's share of AI supercomputer performance has surged from under 40% in 2019 to about 80% by 2025, while the public sector's share has dropped below 20% [8][56] - The United States dominates the global AI supercomputer landscape, accounting for approximately 75% of total performance, followed by China at 15% [10][59] - The shift from public to private ownership of AI supercomputers reflects the growing economic importance of AI and the increasing investment in AI infrastructure [54][68]