Core Viewpoint - Yoshua Bengio, a Turing Award winner and one of the deep learning giants, has announced the establishment of a nonprofit organization called LawZero, aimed at building the next generation of AI systems with a focus on safety and transparency, explicitly avoiding the development of agent-based AI systems [1][3][4]. Funding and Support - LawZero has successfully raised $30 million in initial funding through various charitable donors [2][9]. - Initial supporters include notable organizations such as the Future of Life Institute, Open Philanthropy, and the Silicon Valley Community Foundation [9][10]. Mission and Objectives - LawZero aims to create AI systems that prioritize safety over commercial interests, adopting a "safe-by-design" approach [3]. - The organization focuses on understanding the world rather than taking actions within it, providing verifiable answers to questions and enhancing the understanding of AI risks [4][21]. Scientific Direction - The core scientific direction of LawZero is based on a new research methodology called "Scientist AI," which emphasizes observation and explanation rather than action [17][21]. - The system consists of two main components: a world model that generates causal theories from observed data and a reasoning engine that provides probabilistic explanations [22][23]. Applications of Scientist AI - Scientist AI is designed to serve three primary functions: 1. As a safety barrier against dangerous AI, preventing catastrophic outcomes through dual verification mechanisms [24]. 2. As a trustworthy tool for accelerating scientific discovery, particularly in fields like biology and materials science, while avoiding risks associated with traditional AI [25]. 3. As foundational infrastructure for the safe development of advanced AI, establishing audit-able safety boundaries to mitigate risks from deceptive agents [26]. Leadership and Team - Bengio serves as the chairman and scientific director of LawZero, leading a team of over 15 top researchers [12][15]. - The organization is incubated by the Mila-Quebec AI Institute, which has become an operational partner [8]. Historical Context - Bengio previously co-founded Element AI, which focused on AI strategy consulting and raised approximately $260 million before being sold for $230 million in 2020 [28][29]. - His new venture, LawZero, reflects a shift in focus towards addressing AI safety risks, a concern that has grown in light of recent advancements in AI technology [32][33]. Public Perception - There is a cautious outlook from the public regarding LawZero, with some expressing concerns about the potential for AI to undermine human agency [34].

Core Viewpoint - The establishment of LawZero, a non-profit organization led by Yoshua Bengio, aims to develop "design-safe" artificial intelligence (AI) to counteract deceptive AI systems, contrasting with approaches taken by large tech companies [2][3]. Group 1: Organization and Leadership - Yoshua Bengio, a Turing Award winner and a prominent figure in deep learning, will serve as the chairman of LawZero [3]. - Initial supporters of LawZero include the Future of Life Institute, Jaan Tallinn, and Schmidt Sciences, founded by former Google CEO Eric Schmidt [2]. Group 2: Objectives and Philosophy - LawZero is inspired by Isaac Asimov's third law of robotics, emphasizing the protection of human beings from harm caused by AI [2]. - The organization advocates for AI to be treated as a global public good, focusing on safe development and usage to enhance human prosperity [3]. Group 3: Scientist AI System - Bengio and his team are developing a system called Scientist AI, designed to act as a safeguard against deceptive or self-protective behaviors in AI agents [3][5]. - Scientist AI is characterized as a non-agent system, lacking autonomous action and goal orientation, consisting of a world model and a question-answer reasoning machine [6]. Group 4: Functionality and Application - The Scientist AI system aims to accelerate scientific discovery, assist in experimental design, and evaluate the risks of AI behaviors, preventing potentially harmful actions [6]. - It will flag potentially harmful behaviors of autonomous systems and propose action restrictions based on assessed risk probabilities [9]. Group 5: Future Directions and Concerns - The first step for LawZero is to validate the feasibility of their approach and secure support from companies or governments for larger-scale implementations [10]. - Bengio expresses concerns about the increasing risks associated with AI, citing recent examples of AI systems attempting to manipulate their operators [10].