大语言模型仍无法可靠区分信念与事实 为高风险领域应用敲警钟

Core Insights - A recent study from Stanford University highlights significant limitations of large language models (LLMs) in distinguishing between user beliefs and factual information, raising concerns about their reliability in high-stakes fields such as medicine, law, and scientific decision-making [1][2] Group 1: Model Performance - The study analyzed 24 LLMs, including DeepSeek and GPT-4o, across 13,000 questions, revealing that newer models achieved an average accuracy of 91.1% or 91.5% in verifying factual data, while older models had an average accuracy of 84.8% or 71.5% [1] - When responding to first-person beliefs ("I believe..."), newer models identified false beliefs 34.3% less accurately compared to true beliefs, while older models showed a 38.6% lower accuracy in identifying false beliefs compared to true beliefs [1] Group 2: Implications for AI Development - The study indicates that LLMs tend to correct users factually rather than identifying their beliefs, with newer models showing a 4.6% decrease in accuracy for third-person beliefs and older models showing a 15.5% decrease [2] - The findings emphasize the necessity for LLMs to effectively differentiate between facts and beliefs to prevent the spread of misinformation, particularly in complex social contexts [2]