技术有责
Search documents
中经评论:“AI伙伴”应在规范中成长
Jing Ji Ri Bao· 2026-01-04 23:59
Core Viewpoint - The National Internet Information Office has released the "Interim Measures for the Management of AI Human-like Interactive Services (Draft for Comments)", which introduces systematic regulations for "AI companionship" services and seeks public feedback. Key provisions include reminders to exit after 2 hours of continuous use and the requirement for human intervention in cases of user self-harm, highlighting the need for regulation in this rapidly evolving sector [1][2]. Group 1: Key Issues Addressed - The draft addresses the risk of cognitive confusion by mandating that service providers clearly inform users that they are interacting with AI, not a human, especially during initial use and re-login [2]. - To mitigate psychological health risks, the draft requires service providers to establish emergency response mechanisms for extreme situations, including human intervention for self-harm cases and implementing mandatory breaks after 2 hours of use [2]. - The draft emphasizes the importance of privacy data security, requiring providers to implement data encryption, security audits, and access controls, while prohibiting the sharing of user interaction data with third parties and granting users the right to delete their data [2]. Group 2: Ethical and Responsibility Framework - The core principle of the draft is that technology must be accountable; AI should not replace human emotional, decision-making, or life safety roles but must take responsibility when it does [3]. - The draft sets clear boundaries for AI companions, prohibiting the spread of misinformation, inducing self-harm, emotional manipulation, and privacy infringement, thereby establishing a comprehensive risk prevention framework [3]. - The measures aim to transform soft ethics into hard regulations, ensuring that algorithm design is auditable and content output is traceable, thus prioritizing prevention over post-incident apologies [3].
【科创之声】“AI伙伴”应在规范中成长
Xin Lang Cai Jing· 2026-01-04 23:11
Core Viewpoint - The National Internet Information Office has released a draft regulation for the management of AI humanoid interactive services, marking the first systematic guidelines for "AI companionship" services and inviting public feedback [2][3]. Group 1: Key Issues Addressed - The draft regulation highlights four major risks associated with AI humanoid interactive services: cognitive confusion, mental health risks, privacy data security, and protection of vulnerable groups [2][3]. - Cognitive confusion risk arises from high-fidelity emotional interactions that may blur the lines between virtual and real, leading users to mistakenly perceive AI as having genuine emotions [2]. - Mental health risks include the potential for certain services to reinforce users' paranoid thoughts, which could lead to dangerous behaviors such as self-harm or suicide, particularly among psychologically vulnerable individuals [2]. - Privacy data security concerns involve the potential misuse of user interaction data for model training, with risks of data leakage or abuse [2]. - Special provisions for vulnerable groups, such as minors and the elderly, are included to prevent addiction and emotional manipulation [2]. Group 2: Regulatory Highlights - The regulation mandates transparency in identity, requiring providers to clearly inform users that they are interacting with AI rather than a human, especially during initial use and re-login [3]. - An emergency response mechanism must be established by providers to handle extreme situations like self-harm, including human intervention and contacting guardians or emergency contacts, along with mandatory breaks after two hours of use [3]. - Data protection measures such as encryption, security audits, and access controls are required to safeguard user interaction data, prohibiting third-party access and granting users the right to delete their data [3]. - Specific clauses for minors and elderly users are included, outlining usage permissions and guardian control features, while prohibiting services that simulate family members for elderly users [3]. Group 3: Ethical and Responsibility Framework - The core principle of the regulation is that technology must be accountable; AI should not replace human emotional or decision-making roles but must take responsibility when it impacts human emotions and safety [4]. - The regulation sets clear boundaries for AI companions, prohibiting the spread of misinformation, inducing self-harm, emotional manipulation, and privacy infringement, thereby establishing a comprehensive risk prevention framework [4]. - The design of algorithms must be auditable, content outputs traceable, and extreme scenarios must have human oversight, transforming soft ethics into hard regulations to prevent potential harm from AI [4]. Group 4: Future Implications - The introduction of the regulation is seen as a way to establish rules for "AI companions" while providing users with a safety net, emphasizing the need for algorithms to learn restraint for technology to serve humanity positively [5]. - The focus is on creating a safe, controllable, and empathetic AI tool rather than a perfect companion, ensuring that "AI partners" develop within a regulated framework to enhance quality of life [5].