陶哲轩谈 AI：最危险的不是不会，是“看起来没错”

Core Viewpoint - The main concern raised by Fields Medalist Terence Tao is that the danger of AI lies not in its inability to perform tasks, but in its tendency to produce outputs that appear correct while being fundamentally flawed [2][14]. Group 1: Mimicry - Current AI can generate seemingly valid mathematical proofs but lacks true understanding, often producing nonsensical answers when questioned [7][9]. - AI's outputs are based on statistical probabilities rather than logical reasoning, leading to a false sense of confidence in its correctness [13]. - The phenomenon of "Contamination" occurs when AI repeats training data rather than genuinely deriving conclusions, highlighting its lack of judgment [10][12]. Group 2: Motivation - AI lacks the ability to assess the significance of problems, which is crucial for making value judgments in mathematics [16][21]. - Unlike humans, AI cannot determine which problems are worth solving or which theorems are critical, limiting its effectiveness in generating innovative solutions [18][20]. - The essence of AI is its ability to recall known information, but it cannot discern what is important or valuable [22]. Group 3: Verification - AI-generated outputs often fail to pass verification checks, as the process of deriving conclusions is not transparent [23][26]. - In various fields, such as law and programming, reliance on AI without proper validation has led to significant errors and consequences [29]. - The recommendation is to use AI only within verifiable limits, pairing it with human or automated verification systems to ensure accuracy [30][34]. Group 4: Proper Use of AI - AI's true value lies in handling a large volume of medium-difficulty problems that are not prioritized by top researchers, thus addressing a significant gap in research capacity [32]. - AI can assist in literature reviews and data analysis, but it must be used as a tool for generating leads rather than providing final answers [33]. - The key takeaway is to trust AI outputs but always verify their correctness, emphasizing the importance of human oversight in decision-making processes [36][42].