Company and Industry Summary Company: 英维克 (Invec) Key Points - **AWS Factory Audit Progress**: The audit progress and feedback from AWS have exceeded market expectations, indicating strong confidence in the company's capabilities [1] - **Trainium3 Product Upgrade**: The AWS Trainium3 product is set to fully adopt a liquid cooling solution by 2026, representing a significant upgrade over previous generations and better meeting the thermal management needs of high-performance computing scenarios [1][3] - **Liquid Cooling Market Growth**: The standardization plan for liquid cooling is expected to be a core driver of growth in the overseas liquid cooling market, highlighting the increasing demand for advanced cooling solutions [2] - **Order Performance**: The company has exceeded expectations for annual orders, with potential for further upward revisions. The anticipated shipments from foreign CSP manufacturers for TPU have been adjusted upwards, contributing to a sustained increase in liquid cooling penetration [3][4] - **Domestic Orders**: Domestic orders are expected to stabilize between 1.5 to 2 billion yuan. The previous market expectation was around 5 billion yuan, but the actual order scale is now confirmed to be at least 6.5 to 7 billion yuan, with potential for further upward adjustments, providing solid support for performance exceeding expectations [4] - **NVIDIA Collaboration**: There is a high probability that the company will secure the code for the Rubin200 microchannel liquid cooling plate from NVIDIA. The company possesses comprehensive advantages in technology, supply chain, and customer service in the liquid cooling sector [4] - **Future Growth Potential**: As orders materialize, the scale is expected to be adjusted upwards, further enhancing performance [5]

Core Viewpoint - Microsoft has announced the launch of its self-developed AI inference chip, Maia 200, claiming it to be the highest-performing self-developed chip in all large-scale data centers, aimed at significantly enhancing the economic efficiency of AI token generation [5]. Technical Specifications - Maia 200 is manufactured using TSMC's 3nm process and features over 140 billion transistors, with a memory subsystem that includes 216GB of HBM3e and a read/write speed of 7TB/s [5]. - The chip is designed for low-precision computing models, providing over 10 PFLOPS performance at FP4 precision and over 5 PFLOPS at FP8 precision, all within a 750W SoC TDP range [5]. - Its FP4 performance exceeds that of Amazon's AWS Trainium3 by more than three times, while its FP8 performance surpasses Google's TPU v7 [6]. Memory and Interconnect - The redesigned memory subsystem focuses on narrow precision data types and includes a dedicated DMA engine and on-chip SRAM, enhancing token throughput [8]. - Maia 200 offers a bidirectional bandwidth of 2.8TB/s, which is higher than AWS Trainium3's 2.56TB/s and Google's TPU v7's 1.2TB/s [9]. Performance and Efficiency - Maia 200 is the most efficient inference system deployed by Microsoft to date, with a performance improvement of 30% per dollar compared to the latest generation of hardware currently in use [10]. - The chip can run the largest models available today and is designed to support future models, including OpenAI's latest GPT-5.2 [11][12]. Integration and Development - Maia 200 integrates seamlessly with Microsoft Azure, and a software development kit (SDK) is in preview, providing tools for building and optimizing models [13]. - The architecture simplifies programming and enhances workload flexibility while reducing idle capacity, maintaining consistent performance and cost-effectiveness at cloud scale [21][22]. Deployment and Scalability - The deployment time for Maia 200 chips is halved compared to similar AI infrastructure projects, allowing AI models to run shortly after the first chips arrive [23]. - The architecture is designed for scalable performance in dense inference clusters while reducing power consumption and total cost of ownership for Azure's global clusters [22]. Future Outlook - Microsoft is positioning Maia 200 as a solution for the next generation of AI systems, aiming to set new benchmarks for performance and efficiency in critical AI workloads [28]. - The company invites developers, AI startups, and academia to explore early model and workload optimization using the new Maia 200 SDK [29].

Core Insights - Microsoft has launched its self-developed AI inference chip, Maia 200, claiming it to be the highest-performing self-developed chip in all large-scale data centers, aimed at significantly enhancing the economic efficiency of AI token generation [1] Group 1: Chip Specifications - Maia 200 is manufactured using TSMC's 3nm process and features over 140 billion transistors, with a redesigned memory subsystem that includes 216GB HBM3e (with read/write speeds of up to 7TB/s) and 272MB on-chip SRAM [1][2] - The chip is designed for low-precision computing, providing over 10 PFLOPS performance at FP4 precision and over 5 PFLOPS at FP8 precision, all within a 750W SoC TDP [1] - Its FP4 performance exceeds that of Amazon's AWS Trainium3 by more than three times, while its FP8 performance surpasses Google's TPU v7 [1][2] Group 2: Memory and Interconnect - Maia 200's memory subsystem is optimized for narrow precision data types, featuring a dedicated DMA engine and a specialized on-chip network architecture to enhance token throughput [2] - The chip offers a bidirectional bandwidth of 2.8TB/s, outperforming AWS Trainium3's 2.56TB/s and Google TPU v7's 1.2TB/s [3] Group 3: Performance and Efficiency - Maia 200 is Microsoft's most efficient inference system to date, achieving a 30% improvement in performance per dollar compared to the latest generation of hardware currently deployed by Microsoft [3] - The chip can run the largest models available today and is designed to support future models, including OpenAI's latest GPT-5.2, enhancing the cost-effectiveness for Microsoft Foundry and Microsoft 365 Copilot [4] Group 4: Integration and Deployment - Maia 200 integrates seamlessly with Microsoft Azure, and a software development kit (SDK) is in preview, providing tools for building and optimizing models on Maia 200 [6] - The chip's deployment time is reduced by more than half compared to similar AI infrastructure projects, leading to higher resource utilization and faster production delivery [10] - The architecture allows for scalable performance while reducing power consumption and total cost of ownership for Azure's global clusters [9][12] Group 5: Future Outlook - Microsoft is positioning Maia 200 as a foundational element for future generations of AI systems, inviting developers and researchers to explore early model and workload optimization using the new SDK [13]

Core Concepts - The article emphasizes the importance of performance per total cost of ownership (Perf per TCO) and operational flexibility in the design and deployment of AWS Trainium3 [4][8] - AWS adopts a multi-source component supplier strategy and custom chip partnerships to optimize TCO and accelerate time to market [4][8] AWS Software Strategy - AWS is transitioning from internal optimization to an open-source ecosystem, aiming to leverage contributions from external developers to enhance its software offerings [5][10] - The strategy includes releasing and open-sourcing new native PyTorch backends and developing an open software stack to expand AWS's ecosystem [5][10] Market Competition Landscape - The competitive landscape for Trainium3 includes major players like NVIDIA, AMD, and Google, with AWS needing to accelerate development to maintain its market position [7][10] - Trainium3's market strategy focuses on delivering strong performance per TCO and supporting a wide range of machine learning workloads [7][10] Hardware Specifications and Generational Comparison - Trainium3 features significant upgrades over its predecessor, Trainium2, including a doubling of performance metrics and increased memory capacity [12][11] - The article highlights the confusion caused by inconsistent naming conventions in AWS's product lineup and calls for clearer naming similar to NVIDIA and AMD [12][11] Architectural Evolution - The architecture of Trainium3 has evolved to include switched scale-up rack types, which provide better performance and flexibility compared to previous toroidal designs [25][26] - The article details the physical layout and key features of Trainium3's rack architecture, emphasizing its design philosophy focused on maintainability and reliability [27][28] Packaging and Manufacturing Technology - Trainium3 utilizes advanced packaging technologies such as CoWoS-R, which offers cost advantages and improved mechanical flexibility compared to traditional silicon interposers [18][19] - The manufacturing challenges associated with the N3P process node are discussed, highlighting the need for careful management of leakage and yield issues [15][20] Commercialization Acceleration Strategies - AWS is implementing strategies to enhance assembly efficiency, including a cableless design and the use of retimers to optimize supply chain management [43][44] - The company aims to adapt to data center readiness and accelerate commercialization through flexible deployment options [43][44] Network Architecture and Scalability - The article outlines the network architecture of Trainium3, focusing on its horizontal and vertical scaling capabilities, which are designed to optimize performance for machine learning tasks [48][49] - AWS's strategy includes minimizing total cost of ownership while maximizing flexibility in network switch options [48][49]

Core Insights - The AI server design is undergoing a structural transformation, with the transition to cableless architecture and high-density interconnect (HDI) designs becoming central to the PCB industry's evolution [1][2] - The introduction of the Rubin platform marks a significant shift in PCB's role, emphasizing signal integrity and transmission stability as core design metrics [1][2] Group 1: PCB Design and Technology - The Rubin platform utilizes a cableless interconnect design, enhancing the PCB industry's status by shifting from traditional cable-based connections to multi-layer PCBs [1] - The new design materials include M8U grade for Switch Tray and M9 for Midplane, with PCB value per server increasing by over two times compared to previous generations [2] - The design logic of Rubin has become a common language in the industry, influencing other ASIC AI servers like Google TPU V7 and AWS Trainium3 [2] Group 2: Material Innovations - The demand for PCB performance in AI servers is driving significant changes in upstream materials, focusing on dielectric and thermal stability [2] - Nittobo is investing 15 billion yen to expand production of T-glass, which is expected to triple its capacity by the end of 2026, becoming a core material for ABF and BT substrates [2] - Low roughness HVLP4 copper foil is becoming mainstream due to the increasing impact of skin effect, leading to long-term supply tightness and a shift in bargaining power back to upstream material suppliers [3]

Core Viewpoint - The article discusses the significance of Nvidia's Rubin CPX, highlighting its tailored design for AI model inference, particularly addressing the inefficiencies in hardware utilization during the prefill and decode stages of AI processing [1][2][3]. Group 1: AI Inference Dilemma - The key contradiction in AI large model inference lies between the prefill and decode stages, which have opposing hardware requirements [2]. - Prefill requires high computational power but low memory bandwidth, while decode relies on high memory bandwidth with lower computational needs [3]. Group 2: Rubin CPX Configuration - Rubin CPX is designed specifically for the prefill stage, optimizing cost and performance by using GDDR7 instead of HBM, significantly reducing BOM costs to 25% of R200 while providing 60% of its computational power [4][6]. - The memory bandwidth utilization during prefill tasks is drastically improved, with Rubin CPX achieving 4.2% utilization compared to R200's 0.7% [7]. Group 3: Oberon Rack Innovations - Nvidia introduced the third-generation Oberon architecture, featuring a cable-free design that enhances reliability and space efficiency [9]. - The new rack employs a 100% liquid cooling solution to manage the increased power demands, with a power budget of 370kW [10]. Group 4: Competitive Landscape - Nvidia's advancements have intensified competition, particularly affecting AMD, Google, and AWS, as they must adapt their strategies to keep pace with Nvidia's innovations [13][14]. - The introduction of specialized chips for prefill and potential future developments in decode chips could further solidify Nvidia's market position [14]. Group 5: Future Implications - The demand for GDDR7 is expected to surge due to its use in Rubin CPX, with Samsung poised to benefit from increased orders [15][16]. - The article suggests that companies developing custom ASIC chips may face challenges in keeping up with Nvidia's rapid advancements in specialized hardware [14].

Investment Rating - The industry investment rating is classified as In-Line [8]. Core Insights - The report addresses the mismatch in AWS Trainium2/2.5 chip shipments attributed to unstable PCB yield rates, with an expectation of approximately 1.1 million chip shipments in 2025 [1][3]. - Supply chain checks estimate total shipments for the Trainium2/2.5 life cycle (2H24 to 1H26) at 1.9 million units, with a focus on production and consumption in 2025 [2][11]. - The report highlights a significant gap between upstream chip production and downstream consumption, suggesting improvements in yield rates may reduce this gap by 2H25 [6][11]. Upstream - Chip Output Perspective - As of late 2024, 0.3 million units of Trainium2 chips were produced, with a projected total of 1.1 million shipments in 2025, primarily packaged by TSMC (70%) and ASE (30%) [3][11]. - An additional 0.5 million Trainium2.5 chips are expected to be produced in 1H26, bringing the total life cycle shipments to 1.9 million units [3]. Midstream - PCB Perspective - Downstream checks indicate potential shipments exceeding 1.8 million units of Trainium chips, averaging around 200K per month since April [4][11]. - Key suppliers for PCB boards include Gold Circuit and King Slide, which provide essential components for Trainium computing trays [4]. Downstream - Server Rack System Perspective - Wiwynn is identified as a key supplier for server rack assembly, with revenue from AWS Trainium2 servers increasing in 1Q25, aligning with the upstream chip production estimates [5][11]. - The report notes that each server rack can accommodate 32 chips, supporting the projected consumption figures [5]. Component Suppliers - Major suppliers for Trainium2 AI ASIC servers include AVC for thermal solutions, Lite-On Tech for power supply, and Samsung for memory components [10][18]. - Other notable suppliers include King Slide for rail kits and Bizlink for interconnect solutions [10][18]. Future Projections - For Trainium3, shipments are estimated at 650K for 2026, with production managed by Alchip [12][13]. - The report anticipates that Trainium4 will enter small production by late 2027, with a rapid ramp-up expected in 2028 [14].