12/5/2025 AWS Trainium3 Deep Dive | A Potential Challenger Approaching NL72×2/NL32×2 Scale Up Rack Architecture, Step-Function Software & System ·公众号· More Than Semi Improvements, Optimized Perf per TCO, "Amazon ... AWS Trainium3: 优化性价比（Perf per TCO）与运营灵活性 核心理念:性价比与灵活性 运营玩 元 最大将性价比 (Perf per TCO) 多源组件供应商 定制芯片合作伙伴 供应链管理 最低 TCO 最快上市时间 硬件北极星指标 避免单一架构承诺,以实现最大适应性。 系统与网络:"Amazon Basics"方法 设计选择 "Amazon Basics" 方法 优化目标 → 带宽扩展交换机 > (12.8T / 25.6T / 51.2T) 为特定客户和数据中心 实现最佳 TCO 的手段, 提供最佳 TCO 而非固定标准。 冷却方式 (液冷 vs 风冷) 纵向扩 ...

Core Insights - Google is positioning itself as a strong competitor to Nvidia by securing significant partnerships and expanding its TPU offerings, potentially disrupting Nvidia's dominance in the AI chip market [1][3] - The shift towards Google's TPU is driven by its system-level cost efficiency and scalability, which appeals to major AI companies like Meta and Anthropic [5][10] - The emergence of a "Google Chain" signifies a structural change in the AI computing landscape, allowing for a more diversified supply chain beyond Nvidia [22][25] Google’s Strategic Moves - Google is negotiating multi-billion dollar TPU purchases with Meta, which may lead to a shift of some of Meta's computing power from Nvidia to Google [1] - A partnership with Anthropic aims to expand TPU capacity significantly, indicating a strong demand for Google's AI infrastructure [1] - Google's TPU is designed to optimize cost and efficiency, with the latest generation showing a performance-to-cost ratio improvement of up to 2.1 times compared to previous models [5][7] Performance Comparison - Nvidia's Blackwell architecture remains the industry benchmark for single-chip performance, but Google is focusing on system-level efficiency rather than direct competition on chip performance [4][5] - Google’s TPU v5e can achieve a performance-to-cost ratio that is 2-4 times better than traditional high-end GPU solutions, making it an attractive option for large model training [7][10] - The cost of using Google’s TPU v5e is significantly lower than Nvidia's H100, with TPU priced at $0.24 per hour compared to H100's $2.25 [8][9] Market Dynamics - The increasing adoption of Google’s TPU by major AI firms indicates a shift in the AI computing market, where companies are looking for alternatives to Nvidia to mitigate risks and reduce costs [10][13] - The competition between "Nvidia Chain" and "Google Chain" is not a zero-sum game; rather, it represents a broader expansion of AI computing resources [22][27] - The structural change allows companies to choose from a diversified set of computing resources based on their specific needs, enhancing flexibility and cost-effectiveness [25][26] Beneficiaries of Google’s Strategy - AVGO is identified as a key player benefiting from Google's TPU ecosystem, providing essential communication and networking components [15][16] - The manufacturing partners, including TSMC, Amkor, and ASE, are crucial for the production of Google's TPU, ensuring the scalability of its offerings [18] - Companies like VRT, Lumentum, and Coherent are positioned to benefit from the increased demand for high-performance cooling and optical communication solutions as TPU deployments expand [20][19] Future Implications - The rise of Google’s TPU could lead to a more balanced and resilient AI infrastructure, reducing the industry's over-reliance on Nvidia [22][25] - The dual-engine approach of Google, combining cloud and edge computing, is expected to reshape the AI landscape, making it more accessible and efficient for various applications [20][21] - The ongoing competition will likely drive further innovation and investment in AI computing, benefiting the entire industry [27]

Report Industry Investment Rating No relevant content provided. Core Viewpoints The report delves into the in - depth analysis of Google clusters, including their Scale - up (3D structure and optical interconnection) and Scale - out aspects, and also compares the architectures of different GPUs such as NVIDIA and AMD [1][2]. Summary by Directory 1. Google Cluster's Scale up: 3D Structure - **TPU Architecture**: The Ironwood architecture of TPU has high - performance computing components like TensorCore, XLU, VPU, etc., and is connected by high - speed ICI. It uses HBM3 and HBM3E memory to achieve scale - up of 9216 chips [11][12]. - **From TPU to TPU Rack**: A TPU Tray contains 4 Ironwood TPUs, and a TPU Rack consists of 16 TPU Trays and 64 TPU chips. The rack has a specific physical structure and cooling system [28][29]. - **Comparison with Other GPUs**: Compares the architectures of NVIDIA (from Hopper to Blackwell) and AMD (from MI350 to MI400) GPUs, highlighting their different interconnect technologies and performance parameters [20][25]. 2. Google Cluster's Scale up Optical Interconnection: Optical Path Switch - **Optical Switch Components**: The optical path switch uses components such as 850nm camera modules, dichroic beam splitters, fiber collimators, and 2D MEMS micromirrors to separate or combine calibration light and signal light [46]. - **TPU SuperPod Structure**: A TPU SuperPod consists of 64 Google racks, divided into 8 groups of 8 racks. It integrates 4096 chips, sharing 256TiB of HBM memory, with a total computing performance of over 1 ExaFLOP. Each group of 8 racks has a CDU for liquid - cooling [60]. 3. TPU Cluster, Proportion of Optical Path Switches and Optical Modules - **TPU V4**: The proportion of optical path switches is 1.1% with 4096 TPUs, and the proportion of optical modules is 1.5 [70][84]. - **TPU V7**: The proportion of optical path switches is 0.52% with 9216 TPUs, and the proportion of optical modules is also 1.5 [75][89]. - **Rack - level Data**: For a single rack, there are 6 * 16 external optical modules, 4 * 16 PCB traces, and 80 copper cables [94]. 4. Google Cluster's Scale out - **Switch Parameters**: The Tomahawk 5 switch has 128 400G ports [103]. - **Communication Outside TPU SuperPod**: Communication outside the TPU SuperPod is carried out through the Data - center Network (DCN), which includes optical circuit switches and physical fibers [106][108]. - **NV Scale - out OCS**: In the NV scale - out, OCS is used in a redundant spine - leaf network structure, which can enhance the resilience of the network [113][114]. - **Comparison of Interconnection Schemes in a 100,000 - card Cluster**: Compares the InfiniBand, NVIDIA Spectrum - X, and Broadcom Tomahawk5 interconnection schemes in terms of switch quantity, optical module quantity, cost, etc. [125].

Core Insights - NVIDIA Corporation (NVDA) is expected to report strong third-quarter fiscal 2026 earnings on November 19, driven by its leadership in artificial intelligence (AI) computing and high-performance datacenter GPUs [1][10] - The company anticipates revenues of $54 billion (+/-2%), reflecting a significant increase in AI adoption across various industries [2][10] - The Zacks Consensus Estimate for earnings is set at $1.24, indicating a year-over-year growth of 53.1% and sequential growth of 18.1% [3] Revenue Projections - NVIDIA's projected third-quarter revenues of $54 billion represent a 55.7% increase year-over-year and a 16.9% rise sequentially [2] - The datacenter segment is expected to generate revenues of $48.04 billion, marking a 56.1% year-over-year increase and a 16.9% sequential rise [5][10] Datacenter Growth - The datacenter business has been a key growth driver, with a 56% year-over-year increase in the second quarter of fiscal 2025, reaching $41.1 billion [4] - Heavy investments in NVIDIA's GPUs for AI systems are fueling this growth, as companies and cloud providers increasingly rely on NVIDIA's technology [5][10] AI Demand and Market Trends - The rise of generative AI is creating a strong demand for high-performance computing, with enterprises rapidly integrating AI into their operations [7] - The global generative AI market is projected to reach $967.65 billion by 2032, growing at a CAGR of 39.6%, highlighting NVIDIA's critical role in AI infrastructure [8] Industry Applications - NVIDIA's chips are utilized across various sectors, including healthcare, automotive, manufacturing, and cybersecurity, enhancing applications like digital assistants and language translation [9]

Summary of TSMC and AI Supply Chain Conference Call Industry Overview - The conference call focuses on the semiconductor industry, particularly TSMC's role in the AI supply chain and its capacity expansion plans for 3nm wafers in response to increasing demand from major AI customers like Nvidia and AMD [1][2][11]. Key Points and Arguments TSMC's Capacity Expansion - TSMC is considering expanding its 3nm wafer capacity by an additional 20,000 wafers per month (kwpm) in Taiwan, which could increase its 2026 capital expenditure (capex) to between US$48 billion and US$50 billion, up from the previously expected US$43 billion [3][12]. - The expansion is driven by strong demand from major customers, particularly Nvidia, which has indicated a need for more capacity during a recent visit by its CEO [2][11]. Constraints and Challenges - The main constraint for TSMC's expansion is the availability of clean room space, as all new clean room facilities are allocated for 2nm expansion. TSMC may relocate some 22nm/28nm production from Fab 15 to free up space for 3nm expansion [3][12]. - There is a noted shortage of 3nm wafers, which has affected several customers, including Nvidia, AMD, and Alchip [11]. CoWoS Capacity and Demand - TSMC's CoWoS (Chip on Wafer on Substrate) capacity is expected to be sufficient to meet the projected demand from Nvidia's Rubin chips, despite concerns about potential bottlenecks in front-end capacity and materials like T-glass [4][18]. - The analysis indicates that the total implied CoWoS consumption for TSMC could reach 629,000 wafers, with significant contributions from partnerships with OpenAI and AMD [21]. Stock Implications - The potential increase in 3nm capex is viewed positively for global semiconductor capital sentiment. Morgan Stanley maintains an "Overweight" rating on TSMC and other related companies, anticipating better growth in AI semiconductors [6]. Customer Demand Breakdown - The demand for TSMC's 3nm node is projected to grow significantly, with estimates of 110-120 kwpm in 2025 and 140-150 kwpm in 2026, potentially reaching 160-170 kwpm with the new expansion [11][13]. - Major customers include Nvidia, AMD, and AWS, with Nvidia expected to account for a substantial portion of the demand [28]. Additional Important Insights - The conference call highlighted the importance of TSMC's strategic decisions regarding capacity allocation and customer relationships, particularly in the context of the rapidly evolving AI landscape [2][4]. - The analysis of power deployment plans indicates a strong correlation between AI chip demand and CoWoS capacity, suggesting that TSMC's ability to meet this demand will be critical for its future growth [18][21]. This summary encapsulates the key discussions and insights from the conference call, focusing on TSMC's strategic capacity expansions and the implications for the semiconductor industry in the context of AI demand.

Core Insights - Samsung Semiconductor announced a partnership with NVIDIA to establish an AI factory, marking a significant step in AI-driven manufacturing [1] - The collaboration aims to integrate AI technology throughout the semiconductor manufacturing process, enhancing efficiency and precision [2] - Samsung's stock rose by 3.27% while NVIDIA's stock fell by 2%, reflecting market reactions to the announcement [1] Group 1: AI Factory Development - The AI factory will utilize over 50,000 NVIDIA GPUs to implement AI across all manufacturing stages, from design to quality control [1][2] - This initiative will create a smart manufacturing platform capable of analyzing and optimizing production environments in real-time [2] - Samsung and NVIDIA have a 25-year history of collaboration, extending from early DRAM support to current wafer foundry partnerships [2] Group 2: Advanced Technology Integration - Samsung plans to leverage NVIDIA's accelerated computing technologies to scale the AI factory and utilize the NVIDIA Omniverse platform for digital twin manufacturing [3] - The integration of NVIDIA cuLitho and CUDA-X libraries has improved optical proximity correction (OPC) capabilities by 20 times, enhancing circuit patterning accuracy [3] - Future developments will include new GPU-accelerated EDA tools in collaboration with NVIDIA and EDA partners [3] Group 3: Robotics and AI Ecosystem - Samsung aims to connect virtual simulations with real-world robotic data, enhancing robots' decision-making and operational capabilities [4] - The company has developed AI models that support over 400 million Samsung devices, integrating advanced inference capabilities into manufacturing systems [4] - NVIDIA's RTX PRO 6000 Blackwell servers are being utilized to advance automation and humanoid robot development [4] Group 4: AI-RAN Technology Collaboration - Samsung is collaborating with NVIDIA and other stakeholders to develop AI-RAN technology, which integrates AI capabilities into mobile network architecture [5] - AI-RAN will enable real-time operations for AI endpoints like robots and drones at edge nodes, facilitating the proliferation of physical AI [5] - The concept validation for AI-RAN has been successfully completed, combining Samsung's software-defined networking with NVIDIA's GPU technology [5]

Summary of NVIDIA 2025 Conference Call Company Overview - **Company**: NVIDIA (NasdaqGS: NVDA) - **Event**: 2025 Conference - **Date**: October 28, 2025 Key Industry Insights - **Artificial Intelligence (AI)**: AI is described as the new industrial revolution, with NVIDIA's GPUs at its core, likened to essential infrastructure like electricity and the Internet [6][11][12] - **Accelerated Computing**: NVIDIA has pioneered a new computing model termed "accelerated computing," which is fundamentally different from traditional computing models. This model leverages parallel processing capabilities of GPUs to enhance computational power [11][14][15] - **Telecommunications**: A significant partnership with Nokia was announced, aiming to integrate NVIDIA's technology into the telecommunications sector, particularly for the development of 6G networks [27][30][31] Core Technological Developments - **NVIDIA ARC**: Introduction of the NVIDIA ARC (Aerial Radio Network Computer), designed to run AI processing and wireless communication simultaneously, marking a revolutionary step in telecommunications technology [28][29] - **Quantum Computing**: NVIDIA is advancing quantum computing by connecting quantum processors directly to GPU supercomputers, facilitating error correction and AI calibration [38][40][41] - **CUDA and Libraries**: The CUDA programming model and various libraries developed by NVIDIA are crucial for maximizing the capabilities of GPUs and enabling developers to create applications that utilize accelerated computing [16][21][22] Financial and Market Position - **Market Growth**: NVIDIA anticipates significant growth driven by the demand for AI and accelerated computing, with projections indicating visibility into half a trillion dollars of cumulative revenue through 2026 [108] - **Investment in Infrastructure**: Major cloud service providers (CSPs) are expected to invest heavily in capital expenditures (CapEx) to adopt NVIDIA's advanced computing technologies, enhancing their operational efficiency [103] Additional Insights - **AI's Role in the Economy**: AI is positioned as a transformative force that will engage previously untapped segments of the economy, potentially addressing labor shortages and enhancing productivity across various industries [63] - **Technological Shifts**: The industry is experiencing a shift from general-purpose computing to accelerated computing, with NVIDIA's GPUs being uniquely capable of handling both traditional and AI workloads [106] Conclusion NVIDIA is at the forefront of several technological revolutions, particularly in AI and accelerated computing, with strategic partnerships and innovative products that position the company for substantial growth in the coming years. The emphasis on collaboration with major players in telecommunications and the advancement of quantum computing further solidifies NVIDIA's role as a leader in the tech industry.

Core Insights - The emergence of a revolutionary new computing model centered around accelerated computing and AI is seen as a pivotal moment in the tech industry, comparable to past innovations like the microprocessor and the internet [1][2][3] - NVIDIA's GPUs are positioned as essential infrastructure for the new industrial revolution driven by AI, with every company and nation expected to adopt this technology [1][2] Group 1: Accelerated Computing - NVIDIA has developed a new computing model that leverages accelerated computing, which is fundamentally different from traditional CPU-based computing, requiring new algorithms and libraries [3][4] - The company has been advancing accelerated computing for 30 years, culminating in the introduction of the CUDA programming model, which allows for efficient use of GPUs [4][5] - Accelerated computing is now recognized as a critical moment in the evolution of computing, as traditional transistor performance has plateaued [3][4] Group 2: AI and Telecommunications - NVIDIA is partnering with Nokia to create the NVIDIA ARC, a new product line designed for 6G telecommunications, integrating AI to enhance wireless communication efficiency [7][8] - The use of AI in radio access networks (RAN) will improve spectral efficiency, which is crucial for managing energy consumption in wireless networks [8][9] - This partnership aims to position the U.S. at the forefront of the next telecommunications revolution, moving away from reliance on foreign technologies [7][8] Group 3: Quantum Computing - NVIDIA is advancing quantum computing by introducing NVQ-Link, an architecture that connects quantum processors with NVIDIA GPUs for error correction and simulation [10][11] - The integration of quantum computing with AI supercomputing is seen as the future of computational science, enabling more complex problem-solving capabilities [10][11] - The Department of Energy is collaborating with NVIDIA to build new AI supercomputers, emphasizing the importance of computing in scientific advancement [12][13] Group 4: AI's Economic Impact - AI is transforming the computing stack, moving from traditional hand-coded software to data-intensive machine learning models that run on GPUs [14][15] - The AI industry is experiencing exponential growth, driven by smarter models that require more computational resources, creating a virtuous cycle of demand and supply [22][23] - AI is expected to engage a broader segment of the economy, enhancing productivity and addressing labor shortages [17][22] Group 5: Future Innovations - NVIDIA is focusing on extreme co-design to innovate across hardware and software, aiming to create systems that can handle the increasing demands of AI applications [24][25] - The introduction of NVLink 72 and the Grace Blackwell architecture is set to revolutionize AI computing, offering significant performance improvements [26][27] - The company anticipates substantial capital expenditures from major cloud service providers, aligning with the launch of its new architectures [28][29]

Core Insights - HAMi is an active open-source project maintained by over 350 contributors from more than 15 countries, adopted by over 200 enterprises and institutions, showcasing its scalability and support capabilities [2] - The introduction of topology-aware scheduling for NVIDIA GPUs in version v2.7.0 addresses communication bottlenecks in high-performance computing (HPC) and AI model training scenarios, optimizing task deployment to enhance overall computational efficiency [2][3] Feature Overview - The core design of HAMi's topology-aware scheduling involves quantifying the physical topology into "communication scores" between devices, allowing the scheduler to make optimal decisions based on these scores [5] - Dynamic calculation of topology scores is facilitated by Device Plugin using NVML to detect physical connections between GPUs, providing a basis for scheduling decisions [6] - The scheduling process consists of two phases: topology registration, which quantifies physical connections into understandable scores, and scheduling decision-making, which selects the optimal devices based on these scores [9][10] Implementation Details - The discovery and quantification of topology information are crucial for subsequent intelligent decision-making, generating a score table for reporting [13] - The Fit function implements a dual-strategy optimization algorithm, ensuring long-term health of cluster topology resources by automatically applying "best match" and "minimal disruption" strategies for multi-GPU and single-GPU tasks respectively [6][22] Usage - Users can enable topology-aware scheduling with a simple annotation, allowing the scheduler to automatically apply the appropriate strategy based on the requested number of GPUs [25][26] - The design philosophy emphasizes dynamic discovery over static configuration and foresighted decision-making over short-sighted allocation, providing a robust GPU scheduling solution for large-scale AI training and HPC tasks in cloud-native environments [27]

Core Insights - Oracle announced the launch of its large-scale cloud AI supercomputer, Oracle Cloud Infrastructure (OCI) Zettascale10, during the global AI conference [1] - OCI Zettascale10 connects tens of thousands of NVIDIA GPUs across multiple data centers, forming a multi-gigawatt cluster [1] - The peak performance of OCI Zettascale10 can reach 16 zettaFLOPS [1]