【科创之声】“AI伙伴”应在规范中成长

Core Viewpoint - The National Internet Information Office has released a draft regulation for the management of AI humanoid interactive services, marking the first systematic guidelines for "AI companionship" services and inviting public feedback [2][3]. Group 1: Key Issues Addressed - The draft regulation highlights four major risks associated with AI humanoid interactive services: cognitive confusion, mental health risks, privacy data security, and protection of vulnerable groups [2][3]. - Cognitive confusion risk arises from high-fidelity emotional interactions that may blur the lines between virtual and real, leading users to mistakenly perceive AI as having genuine emotions [2]. - Mental health risks include the potential for certain services to reinforce users' paranoid thoughts, which could lead to dangerous behaviors such as self-harm or suicide, particularly among psychologically vulnerable individuals [2]. - Privacy data security concerns involve the potential misuse of user interaction data for model training, with risks of data leakage or abuse [2]. - Special provisions for vulnerable groups, such as minors and the elderly, are included to prevent addiction and emotional manipulation [2]. Group 2: Regulatory Highlights - The regulation mandates transparency in identity, requiring providers to clearly inform users that they are interacting with AI rather than a human, especially during initial use and re-login [3]. - An emergency response mechanism must be established by providers to handle extreme situations like self-harm, including human intervention and contacting guardians or emergency contacts, along with mandatory breaks after two hours of use [3]. - Data protection measures such as encryption, security audits, and access controls are required to safeguard user interaction data, prohibiting third-party access and granting users the right to delete their data [3]. - Specific clauses for minors and elderly users are included, outlining usage permissions and guardian control features, while prohibiting services that simulate family members for elderly users [3]. Group 3: Ethical and Responsibility Framework - The core principle of the regulation is that technology must be accountable; AI should not replace human emotional or decision-making roles but must take responsibility when it impacts human emotions and safety [4]. - The regulation sets clear boundaries for AI companions, prohibiting the spread of misinformation, inducing self-harm, emotional manipulation, and privacy infringement, thereby establishing a comprehensive risk prevention framework [4]. - The design of algorithms must be auditable, content outputs traceable, and extreme scenarios must have human oversight, transforming soft ethics into hard regulations to prevent potential harm from AI [4]. Group 4: Future Implications - The introduction of the regulation is seen as a way to establish rules for "AI companions" while providing users with a safety net, emphasizing the need for algorithms to learn restraint for technology to serve humanity positively [5]. - The focus is on creating a safe, controllable, and empathetic AI tool rather than a perfect companion, ensuring that "AI partners" develop within a regulated framework to enhance quality of life [5].