Core Viewpoint - Huawei is set to unveil a breakthrough technology in AI inference at the "2025 Financial AI Inference Application Landing and Development Forum" on August 12, which aims to significantly reduce China's reliance on HBM (High Bandwidth Memory) and enhance the performance of domestic AI large models, addressing a critical gap in China's AI inference ecosystem [1][3]. Group 1: Current Market Context - The global demand for AI inference is experiencing explosive growth, while the core supporting technology, HBM, is monopolized by foreign giants, with over 90% dependency in high-end AI servers [3]. - The domestic replacement rate for HBM is less than 5%, leading to increased costs for large model training and inference, hindering AI application deployment in key sectors like finance, healthcare, and industry [3]. Group 2: Technological Innovations - Huawei's new technology targets the pain points by optimizing advanced storage-computing architecture and integrating DRAM with new storage technologies, aiming to maintain high inference efficiency while significantly reducing HBM usage [3]. - The technology may involve deep collaboration between "hardware reconstruction + software intelligence," potentially creating "super AI servers" that optimize computing, transportation, and storage capabilities [4]. Group 3: Performance Metrics - Huawei's CloudMatrix384 Ascend AI cloud service has validated similar technological pathways, achieving a single card decode throughput of over 1920 Tokens/s and a 10-fold increase in KV Cache transmission bandwidth, with a 50ms latency for each token output [4]. - The EMS elastic memory storage service has enabled significant reductions in NPU deployment numbers and inference latency, showcasing a 50% reduction in deployment and an 80% decrease in the first token inference latency [4]. Group 4: Industry Applications - The financial sector will be the first to implement Huawei's technology, which has already established a mature AI framework, supporting over 75% of major banks in core transformations [5]. - The technology will enhance native AI applications in finance, enabling millisecond-level decision-making in high-frequency trading and supporting real-time interactions for millions of users in intelligent customer service [5]. Group 5: Future Implications - While the ultra-high bandwidth characteristics of HBM (current mainstream HBM3 bandwidth exceeds 819GB/s) are difficult to fully replace in the short term, Huawei's technological approach offers new options for the industry [5]. - Experts suggest that if the technology can balance performance and cost, it may disrupt the industry's reliance on HBM and shift global AI chip development from "hardware stacking" to "architectural innovation" [5].

Core Insights - The Huawei Developer Conference 2025 featured a summit focused on the "CloudMatrix384 Ascend AI Cloud Service," highlighting its role in accelerating AI innovation across industries through overcoming computational, operational, and storage bottlenecks [1][8]. Group 1: AI Infrastructure Standards - The rapid evolution of AI large models presents challenges in computational, operational, and storage capabilities, which are referred to as the "computational wall," "communication wall," and "storage wall" [2]. - The CloudMatrix384 Ascend AI Cloud Service is positioned as a new standard for AI infrastructure, addressing these challenges effectively [2][6]. Group 2: Technical Features of CloudMatrix384 - The service integrates "hardware reconstruction + software intelligence" to create a high-density, high-speed, and efficient AI-native infrastructure [6]. - High-density capabilities are achieved by connecting 384 Ascend NPUs with 192 Kunpeng CPUs through the MatrixLink high-speed network, forming a "super AI server" that supports up to 160,000 nodes [6]. - High-speed communication is facilitated by the MatrixLink architecture, achieving a bandwidth of 2.8 Tb/s and reducing communication latency to nanoseconds [6]. - Efficiency is enhanced through intelligent scheduling, increasing the effective utilization of computational resources by over 50% [7]. Group 3: Industry Applications and Collaborations - The CloudMatrix384 service has been validated across various industries, with companies like Silicon Flow demonstrating significant performance improvements in AI model training and inference [12][15]. - Other companies, including Sina and iFlytek, have reported enhanced efficiency and performance in their AI applications using the CloudMatrix384 service [22]. - The service is expected to integrate deeply into sectors such as e-commerce, social media, entertainment, finance, and automotive, thereby lowering the barriers to AI innovation [22]. Group 4: Future Outlook - The summit served as a platform for showcasing technological achievements and fostering collaboration among industry players, marking the entry of AI infrastructure into the "super node era" [22]. - Huawei Cloud aims to partner with clients and stakeholders to drive industry-wide intelligent transformation [22].