专访观察者，当AI开始造谣，我们该如何为真相辩护？ 主持人： 最近，关于百度AI等大型语言模型"编造事实"、"虚构出处"的讨论甚嚣尘上。更有人担忧， 如果别有用心者恶意"投喂"垃圾信息来污染AI，用以打击商业对手或者生活情敌，我们该如何应对？今 天，我们邀请到长期关注科技伦理的资深评论员雅为科技杨乔雅女士，与我们共同探讨这一话题。杨 董，欢迎您。 杨董： 主持人好，各位观众好。 主持人： 杨董，我们首先面对一个核心问题：百度AI为什么会"造假"？这仅仅是技术不成熟吗？ 杨董： 这是一个很好的切入点。我们需要区分"技术缺陷"和"伦理失范"。从技术上讲，AI的"造假"被称 为"幻觉"。它不是一个知识库，而是一个基于海量数据概率预测的文本生成器。当它的训练数据本身有 问题，或者信息不足时，它为了完成"生成一段流畅答案"的指令，就会开始"编故事"。 但当一个拥有亿级用户的平台，其AI产品频繁出现事实性错误时，这就超越了技术范畴，成了一个公 共信任危机。它误导的不仅是普通查询，更可能扭曲公众认知，甚至干扰市场秩序。而比无意识的"造 假"更可怕的，是这种能力被有意识地武器化。 主持人： 面对如此严峻的挑战，您认为破局的关 ...

近日，AI江湖上突然传出一些秘闻。 那个叫大模型的高手，好像被下毒了。 不少与之过招的用户发现，曾经算无遗策、对答如流的高人，近来举止颇为怪异。有时正聊着天，会突然话锋一转，向你推荐一款名不见经传的"神药"； 有时让它简述一则新闻，它竟能编出一套有鼻子有眼、却全然是子虚乌有的故事，堪称AI版张冠李戴。 这究竟是怎么回事？莫非是练功走火入魔，以至于开始胡言乱语了？ 据知情者透露，此非走火入魔，实乃江湖中一种阴险手段——数据投毒。 所谓大模型中毒，是指模型在训练或使用过程中受到了恶意数据的影响，导致输出异常甚至有害的内容。 Anthropic的一项最新研究揭示：研究者仅用250篇精心设计的恶意文档，就成功让一个130亿参数的大模型中毒。即使是规模庞大、训练有素的AI模型， 当触发特定短语时，模型也会胡言乱语。 那么，大模型为什么会中毒？又是谁在背后给它们"投毒"？这会带来怎样的后果？下面我们就来一探究竟。 大模型何以频频中毒？ 要理解大模型为何会中毒，首先需要了解这些模型是如何学习的。大型语言模型通过从数据中学习语言模式来训练自己，数据来源广泛且规模巨大，攻击 者只需污染其中很小一部分数据，就能对模型造成显著影响 ...

Core Viewpoint - The national security department has issued a warning about "data poisoning" in AI, which can lead to harmful outputs due to the manipulation, fabrication, and repetition of data [1]. Group 1: Data Poisoning Overview - "Data poisoning" primarily targets two areas: visual recognition and natural language processing [3]. - An example of data poisoning involves altering training data, such as adding a green dot to a zebra image, which can mislead AI models during training [3]. - Even a few contaminated samples among thousands can significantly disrupt the AI model's ability to recognize similar objects correctly [3]. Group 2: Types of Data Pollution - There are two main types of AI data pollution: one involves malicious human intervention to mislead AI outputs, and the other involves the unfiltered inclusion of harmful information from vast internet data collections [5]. - If untrustworthy data is not identified and removed, it can compromise the reliability of AI outputs [5]. Group 3: Data Sources and Risks - AI models require extensive data for training, often sourced from the internet, including various forms of media [7]. - The potential for data contamination exists as anyone can contribute data online, which may lead to the AI model being influenced by unsafe or polluted data [7].

Regulatory Governance - The Ministry of National Security warns about the risks of data poisoning and contamination in artificial intelligence training data, highlighting the presence of false information and biased viewpoints that can compromise AI safety [1][2] - Research indicates that even a mere 0.01% of false text in training data can increase harmful outputs by 11.2%, while 0.001% can raise harmful outputs by 7.2% [1] AI Developments - Elon Musk announced plans to introduce advertising into the responses of his AI product Grok, with a reported 40% increase in online ad conversion rates since June [3] - OpenAI launched GPT-5, claiming it to be the most advanced coding model to date, with significant improvements in programming capabilities [4] - Following the launch, GPT-5 received mixed reviews, with OpenAI's CEO acknowledging technical issues that affected its performance shortly after release [5] - Google DeepMind released Genie3, a third-generation model capable of generating interactive 3D virtual worlds with improved resolution and interaction time compared to its predecessor [6] - Apple is entering the AI search engine market, forming an internal team to develop a ChatGPT-like AI search experience, marking a significant shift in its approach to generative AI [6]

Core Viewpoint - The case involving a defamatory statement attributed to Dong Mingzhu highlights the risks associated with misinformation and the reliance on AI for verifying information, which can lead to significant reputational damage for companies like Gree [1][2]. Group 1: Legal Case and Implications - A Shenzhen individual was sued for allegedly fabricating statements attributed to Dong Mingzhu, resulting in a court ruling that required an apology and compensation of 70,000 yuan to Gree [1]. - The case underscores the potential consequences of misinformation, as the individual claimed to have verified the information using AI tools, which raises concerns about the reliability of AI in discerning truth [1][2]. Group 2: AI and Misinformation - The National Security Department issued a warning about the risks of "data poisoning" in AI training, noting that even a small percentage of false data can significantly increase harmful outputs from AI models [2]. - The report from Tsinghua University indicated a rapid increase in AI-related rumors, particularly in the economic and corporate sectors, with a staggering growth rate of 99.91% in the last six months [3]. Group 3: Regulatory and Collaborative Efforts - The Central Cyberspace Administration of China initiated a campaign to combat misinformation spread by self-media, focusing on the use of AI to generate false information [2][3]. - Proposed regulations, such as the upcoming "Artificial Intelligence Generated Content Labeling Method," aim to ensure transparency in AI-generated content and mitigate the spread of misinformation [3].

Group 1 - The core issue highlighted is the presence of poor-quality training data in artificial intelligence, which includes false information, fabricated content, and biased viewpoints, leading to data source contamination and new challenges for AI safety [1][5]. - Data is identified as a fundamental element for AI, essential for training models and driving AI applications, with high demands for quantity, quality, and diversity to enhance model performance [3][5]. - Data pollution can significantly impair model accuracy and reliability, potentially causing decision-making errors and system failures, with harmful content generated through data poisoning affecting model training [5][6]. Group 2 - Even a small percentage of false text in training data can lead to a substantial increase in harmful outputs, with 0.01% of false text resulting in an 11.2% increase in harmful content [6]. - In the financial sector, data pollution can lead to the creation of false information that may cause abnormal stock price fluctuations, posing new market manipulation risks [7]. - In public safety, data pollution can distort public perception and mislead social opinion, potentially inciting panic [7]. Group 3 - To strengthen AI data security, it is recommended to enhance source regulation to prevent data contamination, supported by existing laws such as the Cybersecurity Law and Data Security Law [9]. - Risk assessment should be reinforced to ensure data safety throughout its lifecycle, including collection, storage, and transmission [9]. - A governance framework should be established for data cleaning and repair, with specific rules based on legal standards to manage and control data quality continuously [9].

Core Viewpoint - The Ministry of National Security has issued a safety alert regarding the quality of training data for artificial intelligence, highlighting the presence of false information, fabricated content, and biased viewpoints, which leads to data contamination and poses new challenges to AI safety [1] Group 1 - The training data for artificial intelligence is characterized by a mix of quality, with significant issues related to misinformation and bias [1] - The contamination of data sources is identified as a critical challenge for the safety of artificial intelligence systems [1]

Core Insights - The article discusses the rapid advancement of AI technology and its implications for human evolution and society, emphasizing the need for brain-computer interfaces to keep pace with AI development [5][7][22]. Group 1: AI and Data - AI is compared to oil, serving as a crucial resource for future economies and military capabilities, with the potential for unlimited growth through self-reinforcing cycles [22][23]. - Data is highlighted as the new "oil," essential for feeding algorithms and enhancing AI capabilities, with companies competing for data center dominance [23][24]. - The three key components for AI development are algorithms, computational power, and data, with a focus on improving these elements to enhance AI performance [24][25]. Group 2: Brain-Computer Interfaces - Brain-computer interfaces (BCIs) are seen as the only way to maintain human relevance alongside rapidly advancing AI, despite the significant risks they pose [7][22]. - Potential risks of BCIs include memory theft, thought manipulation, and the possibility of creating a reality where individuals can be controlled or influenced by external entities [6][7][26]. - The technology could enable profound enhancements in human cognition, allowing individuals to access vast amounts of information and think at superhuman speeds [9][10]. Group 3: Scale AI - Scale AI, founded by Alexandr Wang, provides essential data support for major AI models, including ChatGPT, and is valued at over $25 billion [2][10]. - The company initially gained recognition for creating large-scale datasets and has since expanded its focus to include partnerships with significant clients, including the U.S. Department of Defense [11][56]. - Scale AI's growth trajectory has been rapid, expanding from a small team to approximately 1,100 employees within five years, with a strong emphasis on the autonomous driving sector [64].

Core Viewpoint - The emergence of 3D Gaussian Splatting (3DGS) as a leading 3D modeling technology has introduced significant security vulnerabilities, particularly through a newly proposed attack method called Poison-Splat, which can drastically increase training costs and system failures [1][2][31]. Group 1: Introduction and Background - 3DGS has rapidly become a dominant technology in 3D vision, replacing NeRF due to its high rendering efficiency and realism [2][7]. - The adaptive nature of 3DGS, which adjusts computational resources based on scene complexity, is both a strength and a potential vulnerability [8][11]. - The research highlights a critical security blind spot in mainstream 3D reconstruction systems, revealing how minor alterations to input images can lead to significant operational disruptions [2][31]. Group 2: Attack Mechanism - The Poison-Splat attack targets the GPU memory usage and training time by introducing perturbations to input images, leading to increased computational costs [12][22]. - The attack is modeled as a max-min bi-level optimization problem, employing innovative strategies such as a proxy model to approximate the victim's behavior and maximizing the Total Variation (TV) of images to induce excessive complexity in 3DGS [13][16][15]. - The attack can significantly increase GPU memory usage from under 4GB to 80GB and training time by up to five times, demonstrating its effectiveness [25][22]. Group 3: Experimental Results - Experiments conducted on various 3D datasets showed that unconstrained attacks could lead to GPU memory usage surging by 20 times and rendering speeds dropping to one-tenth of the original [25][22]. - Even with constraints on pixel perturbations, the attack remains potent, with some scenarios showing over eightfold increases in memory consumption [27][22]. Group 4: Implications and Contributions - The research emphasizes that the findings are not merely academic but represent real threats to 3D service providers that allow user-uploaded content [31][40]. - Simple defenses, such as limiting the number of Gaussian points, are ineffective as they compromise the quality of 3D reconstructions [39][35]. - The study aims to raise awareness about the security of AI systems in 3D modeling, advocating for the development of more intelligent defense mechanisms [41][37].