Workflow
CompressARC
icon
Search documents
无预训练模型拿下ARC-AGI榜三!Mamba作者用压缩原理挑战Scaling Law
量子位· 2025-12-15 10:33
Core Insights - The article discusses a new research called CompressARC, which introduces a novel approach to artificial intelligence based on the Minimum Description Length (MDL) principle, diverging from traditional large-scale pre-training methods [1][7][48]. Group 1: Research Findings - CompressARC, utilizing only 76K parameters and no pre-training, successfully solved 20% of problems on the ARC-AGI-1 benchmark [3][5][48]. - The model achieved a performance of 34.75% on training puzzles, demonstrating its ability to generalize without relying on extensive datasets [7][48]. - CompressARC was awarded third place in the ARC Prize 2025, highlighting its innovative approach and effectiveness [5]. Group 2: Methodology - The core methodology of CompressARC revolves around minimizing the description length of a specific ARC-AGI puzzle, aiming to express it as the shortest possible computer program [8][10][23]. - The model does not learn a generalized rule but instead seeks to find the most concise representation of the puzzle, which aligns with the MDL theory [8][9][10]. - A fixed "program template" is utilized, which allows the model to generate puzzles by filling in hardcoded values and weights, thus simplifying the search for the shortest program [25][28]. Group 3: Technical Architecture - CompressARC employs an equivariant neural network architecture that incorporates symmetry handling, allowing it to treat equivalent transformations of puzzles uniformly [38][39]. - The model uses a multitensor structure to store high-level relational information, enhancing its inductive biases for abstract reasoning [40][41]. - The architecture is similar to a Transformer, featuring a residual backbone and custom operations tailored to the rules of ARC-AGI puzzles, ensuring efficient program description [42][44]. Group 4: Performance Evaluation - The model was tested with 2000 inference training steps per puzzle, taking approximately 20 minutes for each puzzle, which contributed to its performance metrics [47]. - CompressARC challenges the assumption that intelligence must stem from large-scale pre-training, suggesting that clever application of MDL and compression principles can yield surprising capabilities [48].