腾讯混元AI Infra核心技术开源，推理吞吐提升30%

Core Insights - Tencent's AI Infra team has launched an open-source production-grade high-performance LLM inference core operator library called HPC-Ops, designed to address production environment pain points [1][3] Performance Improvements - The HPC-Ops library has achieved a 30% improvement in QPM for the mixed Yuan model and a 17% improvement for the DeepSeek model [3] - In terms of single operator performance, HPC-Ops has achieved the following enhancements: - Attention performance improved by up to 2.22 times compared to FlashInfer/FlashAttention - GroupGEMM performance improved by up to 1.88 times compared to DeepGEMM - FusedMoE performance improved by up to 1.49 times compared to TensorRT-LLM [3] Future Development Plans - HPC-Ops will focus on developing sparse Attention operators to address memory and computational bottlenecks for long-context large models [3] - The library will expand its quantization strategies to include more options such as 4bit/8bit mixed precision, aiming to balance inference speed and model accuracy [3] - Additionally, HPC-Ops will implement computation-communication collaborative optimization to significantly reduce communication overhead in distributed inference scenarios, supporting the efficient deployment of ultra-large models [3]