Core Viewpoint - The rapid advancement of artificial intelligence (AI) technology is accelerating the development of medical AI to assist doctors and undertake some of their technical labor, raising concerns about the safety and effectiveness of its application [1][2]. Group 1: Establishment of Evaluation Center - The Beijing Municipal Health Commission has established a Medical AI Application Evaluation Center to create a regulatory framework and standards for evaluating medical AI [1][2]. - The center aims to verify the clinical decision-making capabilities and effectiveness of medical AI, ensuring a safety baseline for its application [1]. Group 2: Evaluation Standards and Methodology - The evaluation of medical AI should be as rigorous as that of human doctors, focusing on multiple dimensions such as safety, professionalism, and practicality [2]. - A multi-dimensional assessment framework has been developed, consisting of six core evaluation dimensions: medical compliance and ethics, evidence-based medicine and knowledge, general auxiliary capabilities, specialty diagnosis and treatment quality control, adaptability of treatment processes, and accuracy of treatment decisions, encompassing over 70 specific evaluation tasks [2][3]. - The evaluation center collaborates with key hospitals, research institutions, and authoritative expert teams to construct a high-quality evaluation dataset using clinical cases and the latest clinical guidelines [2]. Group 3: Innovative Evaluation Mechanism - The evaluation system automatically matches tasks based on application types and generates evaluation reports, which are then reviewed by clinical experts [3]. - An AI-based scoring mechanism has been introduced to quantify scores based on diagnostic reasoning, logic, and results, ensuring objective and scientifically credible evaluation outcomes [3]. - The center plans to expand its evaluation services to cover various medical fields, including internal medicine, surgery, and pediatrics, to support the healthy development of the medical AI industry [3].