大模型追逐星辰大海，GPT和Gemini国际天文奥赛夺金

Core Insights - The article discusses the remarkable advancements in artificial intelligence, particularly in large language models (LLMs) like GPT-5 and Gemini 2.5 Pro, which have achieved gold medal performances in the International Olympiad on Astronomy and Astrophysics (IOAA) [4][18]. Group 1: AI Model Performance - GPT-5 and Gemini 2.5 Pro excelled in the IOAA, demonstrating strong reasoning and problem-solving capabilities in astronomy and astrophysics [4][12]. - In the theoretical exams, GPT-5 scored an average of 84.2% while Gemini 2.5 Pro scored 85.6%, outperforming other models by 7 to 25 percentage points [12][13]. - The models achieved gold medal status, with GPT-5 scoring 86.8% in 2025, 89.6% in 2023, and 93.0% in 2022, consistently outperforming the best human participants [19][18]. Group 2: Evaluation Framework - The study introduced a more rigorous evaluation framework for assessing LLMs in scientific research, focusing on complex reasoning and problem-solving rather than simple knowledge recall [9][10]. - The IOAA was chosen as a benchmark due to its ecological validity, covering a wide range of astronomical topics and requiring multi-step reasoning [10][9]. Group 3: Error Analysis - The models showed a significant performance gap between different types of questions, with better accuracy in physics/mathematics problems (67-91%) compared to geometric/spatial problems (49-78%) [26]. - Common errors included conceptual misunderstandings and geometric reasoning challenges, indicating fundamental difficulties in achieving deep physical understanding [26][25].