3 6 Ke·2026-01-04 11:13Core Insights - The report highlights a significant shift in AI inference economics, where the focus has moved from raw chip performance to the intelligence output per dollar spent [1][4][46] - NVIDIA continues to dominate the market, with its GB200 NVL72 outperforming AMD's MI350X by a factor of 28 in throughput [1][5][18] AI Inference Economics - The key metric for evaluating AI infrastructure has transitioned to "how much intelligence can be obtained for each dollar" [4][6][46] - In high-interaction scenarios, the cost per token for DeepSeek R1 can be reduced to 1/15th of other solutions [2][20] Model Architecture - The report discusses the evolution from dense models to mixture of experts (MoE) models, which activate only the most relevant parameters for each token, improving efficiency [9][11][46] - MoE models are becoming the standard for top open-source large language models (LLMs), with 12 out of the top 16 models utilizing this architecture [11][14] Performance Comparison - In terms of performance, the GB200 NVL72 shows a significant advantage over AMD's MI355X, achieving up to 28 times the performance in certain scenarios [18][24][30] - The report indicates that as interaction rates increase, the performance gap between NVIDIA and AMD platforms widens, with NVIDIA's solutions becoming increasingly efficient [30][37] Cost Efficiency - Despite the higher hourly cost of the GB200 NVL72, its advanced architecture and software capabilities lead to a lower cost per token, making it more economical in the long run [20][41][45] - The analysis shows that the GB200 NVL72 can achieve a performance per dollar advantage of approximately 12 times compared to its competitors [42][44] Future Trends - The future of AI models is expected to lean towards larger and more complex MoE architectures, with platform-level design becoming a critical factor for success [46][47] - Companies like OpenAI, Meta, and Anthropic are likely to continue evolving their flagship models in the direction of MoE and inference, maintaining NVIDIA's competitive edge [46]