编码器 - 解码器transformer模型
Search documents
Nature公开谷歌IMO金牌模型技术细节,核心团队仅10人,一年给AI编出8000万道数学题训练
3 6 Ke· 2025-11-13 09:01
Core Insights - Google DeepMind has publicly released the complete technology and training methods behind its IMO gold medal model, AlphaProof, continuing its tradition of transparency in AI research [1][22]. Group 1: Development and Team Structure - The AlphaProof team was relatively small, typically consisting of about 10 members, with additional personnel joining closer to the IMO competition [3]. - The core breakthrough was attributed to IMO gold medalist Miklós Horváth, who developed a method to create various problem variants for training the AI [3][5]. Group 2: Technical Architecture - AlphaProof employs a 3 billion parameter encoder-decoder transformer model as its "brain," designed to understand the current proof state and output strategies and step estimates for completing proofs [8][9]. - The system transforms the mathematical proof process into a game-like environment, utilizing a reinforcement learning framework based on the Lean theorem prover [6]. Group 3: Training Methodology - The training faced challenges in sourcing sufficient mathematical problems, initially pre-training the model on approximately 300 billion tokens of code and math text [11]. - A specialized translation system was developed to convert natural language math problems into a formal language understood by Lean, generating around 80 million formalized problems from 1 million natural language questions [11][14]. Group 4: Performance and Achievements - AlphaProof demonstrated impressive performance at the 2024 IMO, successfully solving three problems, including the most difficult one, with a training time of 2-3 days per problem [19][20]. - The system's ability to generate related problem variants during testing significantly enhanced its problem-solving capabilities [19][17]. Group 5: Future Directions and Limitations - Following its success, DeepMind has opened access to AlphaProof for researchers, who have reported its strengths in identifying counterexamples and proving complex statements [22][23]. - However, limitations were noted when dealing with custom definitions, indicating a dependency on existing concepts within the Mathlib library [24]. - The reliance on the Lean theorem prover presents challenges due to its evolving nature, which may affect AlphaProof's performance in advanced mathematical fields [24].