中山&清华：基于大模型的具身智能系统综述

Core Viewpoint - The article provides a comprehensive overview of embodied intelligence systems based on large models, highlighting their applications, challenges, and future directions in various domains such as home services, healthcare, education, and industry [6][39]. Summary by Sections Perception and Understanding - Embodied intelligence systems utilize sensors like cameras and microphones to receive raw data and interpret it to form environmental awareness. Large models excel in processing multimodal input data, effectively integrating text, images, and audio to capture relationships and extract high-dimensional features for understanding the world [5][6]. - Multimodal models, such as GPT-4V, enhance the understanding of environments by encoding images and text into a shared vector space, facilitating perception and comprehension of user instructions [9]. Control Levels - The control levels of embodied intelligence systems are categorized into demand level, task level, planning level, and action level, each with representative works that demonstrate the application of large models [6][11]. System Architecture - The architecture of embodied intelligence systems includes end-to-end Transformer architectures and combinations of frozen parameter large models with foundational models, allowing for flexible optimization without sacrificing generalization [21][29]. Data Sources - Data sources for training embodied intelligence systems include simulators, imitation learning, and video learning, with simulators providing a controlled environment for rapid data collection and testing [31][32]. Challenges - Key challenges faced by embodied intelligence systems include the scarcity of real-world data, slow inference speeds, and the need for multi-agent collaboration in complex tasks [39][40]. Future Development Directions - Future directions for embodied intelligence systems involve improving data collection methods, optimizing large models for faster inference, enhancing multi-agent collaboration, and expanding applications across various fields [41][44].