机器之心·2025-11-26 01:36Core Insights - The AI industry is facing unprecedented bottlenecks as large model parameters reach trillion-level, with issues such as low efficiency of Transformer architecture, high computational costs, and disconnection from the physical world becoming increasingly prominent [2][4][38] - ZTE's recent paper, "Insights into Next-Generation AI Large Model Computing Paradigms," analyzes the core dilemmas of current AI development and outlines potential exploratory directions for the industry [2][38] Current State and Bottlenecks of LLMs - The performance of large language models (LLMs) is heavily dependent on the scaling laws, which indicate that ultimate performance is tied to computational power, parameter count, and training data volume [4][5] - Building advanced foundational models requires substantial computational resources and vast amounts of training data, leading to high sunk costs in the training process [5][6] - The efficiency of the Transformer architecture is low, with significant memory access demands, and the current hardware struggles with parallel operations in specific non-linear functions [6][7] Challenges in Achieving AGI - Current LLMs exhibit issues such as hallucinations and poor interpretability, which are often masked by the increasing capabilities driven by scaling laws [9][10] - There is ongoing debate regarding the ability of existing LLMs to truly understand the physical world, with criticisms focusing on their reliance on "brute force scaling" and lack of intrinsic learning and decision-making capabilities [9][10] Engineering Improvements and Optimizations - Various algorithmic and hardware improvements are being explored to enhance the efficiency of self-regressive LLMs, including attention mechanism optimizations and low-precision quantization techniques [12][13][14] - Innovations in cluster systems and distributed computing paradigms are being implemented to accelerate training and inference processes for large models [16][17] Future Directions in AI Model Development - The industry is exploring next-generation AI models that move beyond the Next-Token Prediction paradigm, focusing on models based on physical first principles and energy dynamics [24][26] - New computing paradigms, such as optical computing, quantum computing, and electromagnetic computing, are being investigated to overcome traditional computational limitations [29][30] ZTE's Exploration and Practices - ZTE is innovating at the micro-architecture level, utilizing advanced technologies to enhance AI accelerator efficiency and exploring new algorithms based on physical first principles [36][38] - The company is also focusing on the integration of hardware and software to create more efficient AI systems, contributing to the industry's shift towards sustainable development [38]