AI最大的Bug

Core Viewpoint - The article discusses the phenomenon of "hallucination" in AI, explaining that it arises from the way AI is trained, which rewards guessing rather than admitting uncertainty [5][11]. Group 1: AI Hallucination - AI often provides incorrect answers when it does not know the correct information, as it is incentivized to guess rather than remain silent [5][6]. - An example is given where an AI model provided three different incorrect birth dates for a person, demonstrating its tendency to "hallucinate" answers [5][6]. - OpenAI's research indicates that this behavior is a result of a training system that rewards incorrect guesses, leading to a higher score for models that guess rather than those that admit ignorance [7][8]. Group 2: Training and Evaluation - The training process for AI can be likened to a never-ending exam where guessing is the optimal strategy to achieve a higher score [6][7]. - OpenAI compared two models, showing that one model had a higher accuracy but a significantly higher error rate, while the other model was more honest in its responses [7][8]. - The concept of "singleton rate" is introduced, indicating that if an information appears only once in the training data, the AI is likely to make mistakes when judging its validity [9]. Group 3: Limitations and Misconceptions - OpenAI argues that achieving 100% accuracy is impossible due to the inherent uncertainty and contradictions in the world, meaning hallucinations will always exist [10]. - The article emphasizes that hallucination is not an unavoidable flaw but can be controlled if AI learns to admit when it does not know something [10][11]. - It is noted that smaller models may sometimes be more honest than larger models, as they are less likely to guess when uncertain [11]. Group 4: Philosophical Implications - The article raises questions about the nature of human imagination and creativity, suggesting that hallucination in AI may reflect a similar human trait of creating stories in the face of uncertainty [14][15]. - It posits that the ability to create myths and stories is what distinguishes humans from other animals, and this trait may not be a flaw but rather a fundamental aspect of intelligence [14][15]. - The discussion concludes with a contemplation of the future of AI, balancing the desire for accuracy with the need for creativity and imagination [17].