" 将 Key 跟 Value Cache 按照不同的方法压缩，可以让模型不掉 点。 " 作者丨张进 编辑丨 林觉民 目前，不同大模型厂商发布的大语言模型在处理超长上下文方面已经有显著突破，最高的已能支持数百万 Token 的输入，例如 MiniMax-M1、Qwen2.5-1M 系列模型，均支持百万Token（1M）级别的超长上 下文处理能力。 但是这场有关提升大模型上下文长度的"军备赛"依然不会停止，这是一项巨大的工程与效率之战。因为超 长下文为模型智能提供了最广阔的发挥空间——在处理如金融、法律、医疗等领域的长语境任务时表现更 好。所以谁能率先突破更长上下文处理能力，便有机会创造出更大的商业与技术价值。 胡侠团队便针对这一目标提出了一项最新研究方案——"通过有损计算（Lossy Computation）来提高大 语言模型的推理效率"。这项研究的基本思路是，利用大语言模型对来自低精度计算等"有损"操作产生的 噪声具有极强鲁棒性这一特点，主动引入可控的、不损害性能的信息损失，以换取显著的效率提升。 大模型中的"有损计算"是通过有选择地牺牲一部分精度来大幅降低计算或者存储成本，从而提升推理效 率，主要围绕模型 ...

Core Insights - EvolKV achieves superior performance with only 1.5% of the full KV cache budget, significantly reducing inference costs for large language models [1][11][25] - The traditional KV cache methods face challenges with long input texts, leading to increased storage requirements and slower processing [3][4] KV Cache Optimization - Existing KV cache compression methods primarily rely on heuristic approaches, which may not optimally retain task-relevant information [4][9] - EvolKV introduces an evolutionary framework that adaptively allocates KV cache budgets across transformer layers, optimizing for downstream task performance [6][10] Performance Improvements - In various benchmark tests, EvolKV consistently outperforms baseline methods, achieving up to a 13% improvement in the Needle-in-a-Haystack benchmark and maintaining high accuracy in the GSM8K dataset [11][30][25] - The method demonstrates strong adaptability across diverse tasks, maintaining competitive performance even with reduced cache budgets [25][29] Experimental Results - Comprehensive experiments on Mistral 7B-Instruct and Llama-3-8B-Instruct show that EvolKV outperforms all baseline methods across multiple KV cache budget configurations [22][24] - In the LongBench evaluation, EvolKV consistently achieved the highest average performance, even surpassing the full model in certain configurations [22][25] Evolutionary Algorithm Mechanism - The evolutionary algorithm generates candidate solutions and evaluates their fitness based on downstream task performance, guiding the optimization process [13][14] - The optimization process is structured in groups to enhance efficiency, allowing for a more stable optimization dynamic [16][17] Cache Budget Allocation - EvolKV employs a dynamic, task-driven approach to allocate KV cache budgets, ensuring that the distribution aligns with the functional contributions of different transformer layers [10][19] - The method includes a mechanism for adjusting the total KV cache budget to ensure fairness in evaluation [20]