Workflow
具身AGI
icon
Search documents
原神Agent,字节出品
猿大侠· 2025-11-16 04:11
Core Viewpoint - ByteDance has developed a new gaming agent named Lumine, capable of autonomously playing games like Genshin Impact, showcasing advanced skills in exploration, combat, and puzzle-solving [1][4][16]. Group 1: Agent Capabilities - Lumine can perform complex tasks such as dynamic enemy tracking, precise long-range shooting, and smooth character switching, effectively handling various game scenarios [4][6][10]. - The agent demonstrates strong understanding in boss battles and can solve intricate puzzles, indicating high spatial awareness [6][8][10]. - Lumine is capable of executing GUI operations and can follow complex instructions with clear prior information, enhancing its usability in gaming [12][14]. Group 2: Technical Framework - Lumine is built on the Qwen2-VL-7B-Base model, leveraging multimodal understanding and generation capabilities acquired from extensive training on web data [16]. - The agent employs a unified language space for modeling operations and reasoning, facilitating seamless integration of perception, reasoning, and action [16][19]. - Three core mechanisms are designed for Lumine: Observation Space for visual input processing, Hybrid Thinking for decision-making efficiency, and Keyboard and Mouse Modelling for operational commands [19][22][23]. Group 3: Training Process - The training process consists of three phases: pre-training for basic actions, instruction-following training for task comprehension, and decision reasoning training for long-term task execution [25][27][29]. - Lumine-Base model emerges with core capabilities like object interaction and basic combat, while Lumine-Instruct model achieves over 80% success in short tasks [26][28]. - The Lumine-Thinking model can autonomously complete long-term tasks without human intervention, showcasing its advanced planning and reasoning abilities [30]. Group 4: Performance Evaluation - In comparative tests, Lumine-Base shows over 90% success in basic interactions but lacks goal-oriented behavior in untrained areas [39]. - Lumine-Instruct outperforms mainstream VLMs in task completion rates, achieving 92.5% in simple tasks and 76.8% in difficult tasks, demonstrating superior tactical planning [41]. - Lumine-Thinking completes main story tasks in Genshin Impact with a 100% completion rate in 56 minutes, significantly outperforming competitors like GPT-5 [44][45]. Group 5: Industry Implications - The development of gaming agents like Lumine represents a significant step towards creating general-purpose AI capable of operating in complex 3D environments [50][55]. - Companies like Google are also exploring similar paths with their SIMA 2 agent, indicating a broader industry trend towards utilizing gaming scenarios for training AI [52][56]. - The belief in the eventual transition of gaming agents into real-world applications highlights the potential for embodied intelligence in various sectors [56].
迈向通用具身智能:具身智能的综述与发展路线
具身智能之心· 2025-06-17 12:53
Core Insights - The article discusses the development of Embodied Artificial General Intelligence (AGI), defining it as an AI system capable of completing diverse, open-ended real-world tasks with human-level proficiency, emphasizing human interaction and task execution abilities [3][6]. Development Roadmap - A five-level roadmap (L1 to L5) is proposed to measure and guide the development of embodied AGI, based on four core dimensions: Modalities, Humanoid Cognitive Abilities, Real-time Responsiveness, and Generalization Capability [4][6]. Current State and Challenges - Current embodied AI capabilities are between levels L1 and L2, facing challenges across four dimensions: Modalities, Humanoid Cognition, Real-time Response, and Generalization Capability [6][7]. - Existing embodied AI models primarily support visual and language inputs, with outputs limited to action space [8]. Core Capabilities for Advanced Levels - Four core capabilities are defined for achieving higher levels of embodied AGI (L3-L5): - Full Modal Capability: Ability to process multi-modal inputs beyond visual and textual [18]. - Humanoid Cognitive Behavior: Includes self-awareness, social understanding, procedural memory, and memory reorganization [19]. - Real-time Interaction: Current models struggle with real-time responses due to parameter limitations [19]. - Open Task Generalization: Current models lack the internalization of physical laws, which is essential for cross-task reasoning [20]. Proposed Framework for L3+ Robots - A framework for L3+ robots is suggested, focusing on multi-modal streaming processing and dynamic response to environmental changes [20]. - The design principles include a multi-modal encoder-decoder structure and a training paradigm that promotes cross-modal deep alignment [20]. Future Challenges - The development of embodied AGI will face not only technical barriers but also ethical, safety, and social impact challenges, particularly in human-machine collaboration [20].
快讯|2025世界人形机器人运动会8月落地北京;清华、星动纪元开源首个AIGC机器人大模型;亚马逊推出首款触觉机器人Vulcan
机器人大讲堂· 2025-05-08 06:38
Group 1: Events and Competitions - The 2025 World Humanoid Robot Games will take place in Beijing from August 15 to 17, featuring 19 main events including competitive, performance, and scenario-based competitions [1] - The competitive events will include 11 sub-items such as athletics, testing the robots' movement capabilities, while performance events will showcase collaborative abilities through dance [1] - The scenario-based competitions will focus on practical skills in industrial, hospital, and hotel settings, with 6 specific projects [1] Group 2: Technological Innovations - Tsinghua University and Star Motion Era have released the first open-source AIGC generative robot model, VPP, which predicts future actions and enhances strategy generalization [2][4] - The VPP model can switch between different humanoid robots and operates at a control frequency exceeding 50Hz, demonstrating superior performance in benchmark tests [4] - Longmu Valley's intelligent surgical robot can create personalized 3D surgical plans in 5 to 10 minutes using patient CT data, significantly reducing recovery time for patients [6] Group 3: Collaborations and Partnerships - Youlu Robotics has signed a partnership with Shanghai Shenghua Zizhu Bilingual School to integrate advanced robotics technology into education, enhancing students' programming and logical thinking skills [7][10] - The collaboration aims to provide customized robotic teaching solutions, covering various aspects of education including course instruction and competition guidance [10] Group 4: Industry Developments - Amazon has introduced its first tactile robot, Vulcan, designed to enhance operational efficiency in commercial environments by enabling the robot to perceive and interact with its surroundings [11][13] - Vulcan can handle approximately 75% of items in warehouse settings, matching the speed of human workers while improving safety and ergonomics for employees [13]