为何全球关注超级人工智能（连线评论员）

Core Viewpoint - The rapid development of artificial intelligence (AI) raises concerns about the potential risks associated with superintelligent AI, prompting calls for a pause in its development from a significant number of scientists and industry leaders [1][2]. Group 1: Definitions and Distinctions - General AI is characterized by its high generalization ability, approaching human intelligence levels, and has broad application prospects [1]. - Superintelligent AI is defined as surpassing human intelligence in all aspects and potentially developing autonomous consciousness, leading to actions and thoughts that may be incomprehensible and uncontrollable by humans [1]. Group 2: Risks and Challenges - The primary risks associated with superintelligent AI include alignment failure and loss of control, where even minor deviations from human values could lead to catastrophic outcomes due to the amplification of these errors [2]. - The storage of negative human behaviors in network data increases the risk of superintelligent AI learning and replicating these behaviors, heightening concerns about alignment failure and loss of control [2]. Group 3: Governance and Safety Principles - Safety must be the foundational principle in the development of superintelligent AI, ensuring that security measures cannot be compromised for the sake of model performance [3]. - A proactive defense strategy is essential, focusing on continuous updates through an "attack-defense-assessment" process to address typical security issues like privacy breaches and misinformation [3]. Group 4: Global Cooperation and Governance - The development of superintelligent AI should not lead to a "arms race," and global cooperation is necessary to ensure its safety and reliability for all humanity [4]. - The establishment of international bodies, such as the "Independent International Scientific Group on AI" and the "Global Dialogue on AI Governance," is crucial for coordinating AI governance and promoting sustainable development [4][5]. - Countries, especially those with advanced technologies, have a responsibility to prevent reckless development of superintelligent AI that could lead to widespread risks [5].