清华大学最新综述！具身AI中多传感器融合感知：背景、方法、挑战

Core Insights - The article emphasizes the significance of embodied AI and multi-sensor fusion perception (MSFP) as a critical pathway to achieving general artificial intelligence (AGI) through real-time environmental perception and autonomous decision-making [3][4]. Group 1: Importance of Embodied AI and Multi-Sensor Fusion - Embodied AI represents a form of intelligence that operates through physical entities, enabling autonomous decision-making and action capabilities in dynamic environments, with applications in autonomous driving and robotic swarm intelligence [3]. - Multi-sensor fusion is essential for robust perception and accurate decision-making in embodied AI systems, integrating data from various sensors like cameras, LiDAR, and radar to achieve comprehensive environmental awareness [3][4]. Group 2: Limitations of Current Research - Existing AI-based MSFP methods have shown success in fields like autonomous driving but face inherent challenges in embodied AI applications, such as the heterogeneity of cross-modal data and temporal asynchrony between different sensors [4][7]. - Current reviews often focus on single tasks or research areas, limiting their applicability to researchers in related fields [7][8]. Group 3: Structure and Contributions of the Research - The article organizes MSFP research from various technical perspectives, covering different perception tasks, sensor data types, popular datasets, and evaluation standards [8]. - It reviews point-level, voxel-level, region-level, and multi-level fusion methods, focusing on collaborative perception among multiple embodied agents and infrastructure [8][21]. Group 4: Sensor Data and Datasets - Various sensor types are discussed, including camera data, LiDAR, and radar, each with unique advantages and challenges in environmental perception [10][12]. - The article presents several datasets used in MSFP research, such as KITTI, nuScenes, and Waymo Open, detailing their modalities, scenarios, and the number of frames [12][13][14]. Group 5: Perception Tasks - Key perception tasks include object detection, semantic segmentation, depth estimation, and occupancy prediction, each contributing to the overall understanding of the environment [16][17]. Group 6: Multi-Modal Fusion Methods - The article categorizes multi-modal fusion methods into point-level, voxel-level, region-level, and multi-level fusion, each with specific techniques to enhance perception robustness [21][22][23][24][28]. Group 7: Multi-Agent Fusion Methods - Collaborative perception techniques are highlighted as essential for integrating data from multiple agents and infrastructure, addressing challenges like occlusion and sensor failures [35][36]. Group 8: Time Series Fusion - Time series fusion is identified as a key component of MSFP systems, enhancing perception continuity across time and space through various query-based fusion methods [38][39]. Group 9: Multi-Modal Large Language Model (LLM) Fusion - The integration of multi-modal data with LLMs is explored, showcasing advancements in tasks like image description and cross-modal retrieval, with new datasets designed to enhance embodied AI capabilities [47][50].