谢赛宁李飞飞LeCun搞的寒武纪，究竟是个啥？

Core Viewpoint - The article discusses the emergence and significance of "Cambrian-S," a new AI model focused on spatial perception, aiming to enhance how artificial intelligence understands and interacts with the world [2][6][8]. Group 1: Overview of Cambrian-S - Cambrian-S is not about creating silicon-based chips but rather about enabling AI to genuinely perceive the world [2]. - The model excels in multi-modal video processing, particularly in spatial reasoning tasks, achieving state-of-the-art (SOTA) results in short video spatial reasoning [6][41]. - The model's architecture includes a predictive perception module that allows it to anticipate the next frame in a video, improving efficiency and reducing GPU memory consumption [44]. Group 2: Development and Breakthroughs - The development of Cambrian-S followed a series of breakthroughs, including the evaluation of over 20 visual encoders to identify their strengths and suitable application scenarios [11]. - A spatial visual aggregator (SVA) was designed to efficiently integrate multi-source visual features while maintaining high processing quality [11]. - The team created a high-quality training dataset, filtering from 10 million to 7 million entries to enhance model interaction capabilities [13]. - They established the CV-Bench benchmark to address the inadequacies in existing visual capability assessments [15]. - The optimal training strategy was identified, demonstrating that two-stage training and unfreezing visual encoders significantly enhance model performance [17]. Group 3: Concept of Hyper-Perception - The team introduced the concept of "hyper-perception," which emphasizes the need for AI to not only recognize objects but also understand their spatial relationships and predict their future states [20][23]. - This concept is crucial for developing true multi-modal intelligence, as it allows AI to comprehend continuous video sequences rather than isolated images [25]. Group 4: Testing and Performance - The team developed the VSI-SUPER benchmark to evaluate AI's spatial perception capabilities through tasks like long-term spatial memory and continuous counting [26][30]. - Current models, such as Gemini-Live and GPT-Realtime, showed poor performance in these tests, with accuracy rates below 15% for 10-minute videos [31]. - The Cambrian-S model family, with parameters ranging from 0.5 billion to 7 billion, achieved over 30% improvement in spatial memory accuracy compared to open-source models [41][34].