华为 上新“AI黑科技”

Core Viewpoint - The advent of Token economy signifies a shift in AI model training and inference efficiency, with Huawei introducing UCM (Inference Memory Data Manager) to optimize inference response speed, sequence length, and cost [1][4]. Group 1: UCM Features and Benefits - UCM includes three main components: a connector for different engines and computing power, a library for multi-level KV Cache management and acceleration algorithms, and a high-performance KV Cache adapter, enabling a collaborative approach to AI inference [5]. - UCM aims to enhance inference experience by reducing the first token latency by up to 90% through global prefix caching technology, and it can expand the context window for inference tenfold to accommodate long text processing [5][6]. - The intelligent caching capability of UCM allows for on-demand flow between storage media (HBM, DRAM, SSD), improving TPS (tokens processed per second) by 2 to 22 times in long sequence scenarios, thereby significantly lowering the cost per token [6]. Group 2: Industry Application and Collaboration - Huawei is collaborating with China UnionPay to pilot UCM technology in the financial sector, leveraging the industry's advanced IT infrastructure and data-driven opportunities [7]. - In a joint innovation project with China UnionPay, UCM demonstrated its value by increasing large model inference speed by 125 times, enabling rapid identification of customer issues within 10 seconds [10]. - Huawei plans to open-source UCM in September, contributing to mainstream inference engine communities and promoting the development of the AI inference ecosystem [12].