Core Insights - The article discusses the challenges and solutions related to KV cache compression in long-context reasoning for Vision-Language Models (VLM) and Large Language Models (LLM) [1][2][42] - It introduces MixKV, a method that combines importance and diversity in KV cache selection to enhance stability and coverage in compressed contexts [5][13][42] Group 1: KV Cache Challenges - The lengthening of context leads to linear expansion of KV cache, resulting in increased memory usage and bandwidth costs, which negatively impacts throughput [3][5] - Traditional compression methods often focus solely on "importance," neglecting the inherent "semantic redundancy" present in multimodal KV caches, which can lead to instability [5][12] Group 2: Key Findings - The research team visualized the statistical properties of KV, revealing that multimodal inputs exhibit a higher degree of semantic redundancy, indicating a larger compressible space [8][10] - There are significant differences in redundancy levels across different heads within the same model, suggesting a non-uniform distribution of redundancy [10][12] Group 3: MixKV Solution - MixKV aims to retain KV entries that are both important and diverse, thereby reducing the risk of losing semantic coverage due to redundancy [13][23] - The method consists of two scoring steps (importance and diversity) and a head-wise mixing approach to adaptively balance the two factors based on redundancy levels [14][15][16] Group 4: Experimental Results - MixKV demonstrated consistent performance improvements across various benchmarks in multimodal understanding, long-context reasoning, and GUI localization tasks [25][29][37] - The method showed significant efficiency gains, reducing inference latency and peak memory usage under extreme compression conditions [41][42] Group 5: Conclusion - MixKV represents a critical upgrade for KV cache compression in long-context reasoning, emphasizing the need to consider redundancy structures in the design paradigm for scalable deployment of VLMs and LLMs [42]