Workflow
求索具身智能测评基准EIBench
icon
Search documents
北京人形机器人创新中心VLA模型首个通过具身智能国标测试
Bei Ke Cai Jing· 2025-11-13 04:05
Core Insights - The article discusses the release of the "Qiusuo" embodied intelligence evaluation benchmark EIBench, developed by the China Electronics Standardization Institute, which focuses on the technical requirements for embodied intelligent large model systems [1] - The XR-1 model from the Beijing Humanoid Robot Innovation Center is highlighted as the only VLA model to pass the evaluation, receiving the CESI-CTC-20251103 certification, marking it as the first VLA model in the country to achieve this [1] Group 1 - The EIBench benchmark emphasizes data formats, large model safety, and reliability, establishing a standardized evaluation index system based on national standards [1] - The evaluation criteria include a standardized process for reproducible and fair assessments, a comprehensive task library covering complex scenarios, and a set of performance metrics to quantify model capabilities [1] - In terms of safety, the benchmark includes 14 primary indicators such as controllability, robustness, accountability, privacy protection, functional safety, and resilience [1] Group 2 - During testing, the XR-1 was evaluated on three robots: Tiangong 2.0, UR, and Franka, focusing on dual-arm skills such as picking, pushing, pulling, rotating, and inserting [2] - The evaluation also included generalized testing across seven dimensions, including object color, position, posture, environmental brightness, color temperature, background, and interference [2] - Each test collected 40-50 data points, with over 10 real-machine tests conducted for each task and dimension, ensuring standardized and fair evaluation throughout the process [2]