具身智能之心人形机器人交流群来啦！欢迎从事人形运控、VLA模型、数采、硬件等相关方向的同学 加入。 添加小助理微信AIDriver005，备注昵称+人形+加群。注意：有备注才能通过哦~ ...

Core Viewpoint - Long-VLA is the first end-to-end VLA model specifically designed for long-horizon tasks in robot manipulation, addressing the skill chain problem by introducing phase-aware input masks to dynamically adjust visual modalities during different task phases [2][4][14]. Technical Introduction - Existing technologies for long-horizon tasks can be categorized into three types: end-to-end unified models, task decomposition methods, and input-adaptive modular methods, each with limitations in handling long and complex tasks [3][4]. - Long-VLA combines the advantages of task decomposition within a unified architecture and dynamically adjusts perception modalities through input-level masking, effectively addressing the skill chain issue [4][6]. Model Description - Long-VLA's core design includes three key components: task phase division, input-level adaptation strategy, and unified end-to-end training. Tasks are divided into "movement phases" and "interaction phases," with a newly annotated L-CALVIN dataset to support this division [6][8]. - The input adaptation strategy employs a binary masking mechanism to dynamically adjust attention inputs, enhancing task continuity and mitigating phase distribution differences [6][8]. Experimental Results - In the optimized CALVIN environment, Long-VLA significantly outperformed baseline models in long-horizon tasks, demonstrating stability across ten consecutive sub-tasks [8][10]. - In real-world scenarios involving sorting and cleaning tasks, Long-VLA showed superior performance under varying conditions, confirming its robustness and generalization capabilities [10][12]. - Long-VLA achieved an average task length improvement over baseline methods, with notable increases in performance metrics [13]. Conclusion - This research establishes a balance between end-to-end training and long-horizon adaptability, laying the groundwork for further exploration in robot long-horizon task execution [14].

Core Viewpoint - The article emphasizes the importance of open-source ecosystems in accelerating the development and implementation of embodied intelligence, highlighting collaborations among various industry players and developers [1]. Group 1 - The collaboration between Starry Sea Map and Hugging Face aims to foster a vibrant developer community and explore open-source models and datasets [1][2]. - A live discussion featuring Thomas Wolf, co-founder of Hugging Face, and Zhao Xing, chief scientist of Starry Sea Map, will take place to discuss the future of embodied intelligence and the open-source ecosystem [3][6]. - The live event is scheduled for August 29 at 19:00 [4][10].

Core Insights - Goal-Oriented Navigation (GON) empowers robots to autonomously navigate and complete tasks based on goal descriptions, marking a significant shift from traditional Visual Language Navigation (VLN) systems [2][3] - The technology has been successfully implemented across various sectors, including delivery, healthcare, and hospitality, enhancing service efficiency and adaptability in dynamic environments [3][4] - The evolution of GON technology can be categorized into three generations, each with distinct methodologies and advancements [5][7][9] Group 1: Technology Overview - GON is a key area within embodied navigation, relying on language understanding, environmental perception, and path planning [2] - The transition from following explicit instructions to autonomous decision-making involves semantic parsing, environmental modeling, and dynamic decision-making [2][3] - The integration of computer vision, reinforcement learning, and 3D semantic understanding is crucial for the success of GON systems [2] Group 2: Industry Applications - GON technology has been applied in terminal delivery scenarios, enabling robots to navigate complex urban environments effectively [3] - Companies like Meituan and Starship Technologies have deployed delivery robots that utilize dynamic path re-planning capabilities [3] - In healthcare and hospitality, companies such as Aethon and Jiakan Technology have implemented service robots for autonomous delivery of medications and meals, improving response efficiency [3] Group 3: Technological Evolution - The first generation of GON focused on end-to-end methods using reinforcement and imitation learning, achieving breakthroughs in Point Navigation and closed-set image navigation tasks [5] - The second generation introduced modular methods that explicitly construct semantic maps, enhancing performance in zero-shot object navigation tasks [7] - The third generation integrates large language models (LLMs) and visual language models (VLMs) to improve exploration strategies and open-vocabulary target matching accuracy [9] Group 4: Educational Initiatives - A new course has been developed to address the challenges of learning GON, focusing on practical applications and theoretical foundations [10][11] - The curriculum includes modules on semantic navigation frameworks, Habitat simulation ecology, and end-to-end navigation methodologies [15][18] - The course aims to provide a comprehensive understanding of GON, enabling participants to bridge the gap between theory and practice [11][12]

Core Viewpoint - The article discusses the limitations of traditional Vision-Language-Action (VLA) models and introduces FlowVLA, a new framework that addresses these issues by implementing a Visual Chain of Thought (Visual CoT) principle, enhancing the model's ability to predict future frames through structured physical reasoning rather than mere pixel replication [5][8][36]. Group 1: Background and Current State - VLA models, particularly those pre-trained as world models, show significant potential in the field of general robotics, primarily through large self-regressive Transformers that learn environmental dynamics from vast video data [6][7]. - Existing models face critical flaws, including task confusion leading to prediction failures, knowledge transfer inefficiencies between passive observation and active control, and entangled learning of dynamics and appearance [7]. Group 2: Contributions of FlowVLA - FlowVLA introduces a new learning framework that emphasizes structured physical reasoning by requiring the model to infer motion dynamics before predicting future frames [8][10]. - The model is designed to unify appearance and motion reasoning within a single self-regressive Transformer, maintaining parameter efficiency and architectural simplicity [9][10]. - Experimental results validate FlowVLA's superior performance across various robotic operation benchmarks, demonstrating enhanced sample efficiency and bridging the gap between pre-training and policy fine-tuning [10][20]. Group 3: Research Content - The Visual CoT reasoning process decomposes the frame prediction into a causal chain of "current frame → optical flow → future frame," allowing the model to separate dynamic and appearance learning [12][14]. - The two-phase training paradigm consists of a pre-training phase focused on world model learning and a fine-tuning phase for adapting to control tasks [15][16]. Group 4: Experimental Analysis - FlowVLA outperforms existing methods in the LIBERO dataset across all task sets, particularly excelling in long-term tasks, showcasing its robust understanding of physical dynamics [20][21]. - In the SimplerEnv dataset, FlowVLA demonstrates strong adaptability to visual domain shifts, achieving significant performance improvements in tasks where other models struggle [22][23]. - The model's sample efficiency is validated, requiring only one-third of the training steps to reach peak performance compared to baseline models, with a 55% higher peak success rate in low-data scenarios [30][32]. Group 5: Key Component Validation - Ablation studies on the LIBERO-10 benchmark highlight the importance of the Visual CoT structure, flow loss, and interleaved sequence format, confirming their critical roles in the model's performance [33][34]. Group 6: Comparison with Related Work - FlowVLA distinguishes itself from traditional VLA models by prioritizing dynamic understanding and establishing a robust world model before adapting to control tasks, thus laying a solid foundation for physical knowledge [35].

Core Viewpoint - The article discusses the vision and mission of Zhujidi Power, a company focused on embodied intelligence robotics, aiming to make robots easier to deploy and use in various applications [2][7]. Group 1: Company Overview - Zhujidi Power aims to be a platform for robotics, similar to Nvidia, enabling developers across different fields to create applications and products [3][4]. - The founder, Zhang Wei, emphasizes that while hardware manufacturing for humanoid robots is relatively easy, the real challenge lies in developing the AI capabilities that control them [5][34]. - The company's goal is to create a humanoid robot that is functional, easy to program, and part of a larger ecosystem, akin to a Windows operating system for robots [8][106]. Group 2: Technological Development - The company has made significant advancements in AI-driven control systems, which are crucial for the operation of humanoid robots [39][41]. - Zhang Wei notes that the AI capabilities for controlling robots have matured recently, allowing for more effective movement and task execution [38][40]. - The focus is on developing a robust "small brain" AI that can handle complex tasks, which is seen as a key differentiator in the industry [26][53]. Group 3: Market Positioning and Strategy - Zhujidi Power positions itself as a technology platform that provides foundational capabilities and tools for developers, rather than competing directly in every market segment [76][78]. - The company aims to lower the barriers for developers by reducing hardware costs and providing user-friendly software tools, allowing them to focus on business logic rather than technical complexities [82][84]. - The pricing strategy for their humanoid robot, LimX Oli, is competitive, with a focus on delivering value while ensuring sustainability in operations [114][120]. Group 4: Future Goals and Challenges - The company plans to enhance the usability of its humanoid robots, making them more accessible for programming and integration into various applications [151][152]. - Zhang Wei identifies the biggest challenges as synchronizing technological maturity, ecosystem development, and market acceptance [158]. - The long-term vision includes creating an application ecosystem for their robots, enabling users to easily deploy various functionalities [112][154].

Group 1 - The establishment of the Embodied Intelligence Heart Technology Exchange Group focuses on various advanced technologies including VLA, VLN, remote operation, Diffusion Policy, reinforcement learning, VLA+RL, sim2real, multimodal large models, simulation, motion control, target navigation, mapping and localization, and navigation [1] - Interested individuals can add the assistant's WeChat AIDriver005 to join the community [2] - To expedite the group entry process, it is advised to include a note with the institution/school, name, and research direction [3]

最近越来越多的同学开始给具身智能之心传递好消息，秋招拿到口头offer了、社招成功从自驾转到 具身了。而且据我们了解，对应的薪资和前景都是非常好的。 除此之外，还有很多具身机器人公司委托我们结合EDU版本的本体开发更多的教程与功能。目前已 经在筹备了，后面将慢慢把这类教程公布到我们的具身社区中，促进行业的发展。 "具身智能之心知识星球"是我们一直在维护的具身社区，目前集视频 + 图文 + 学习路线 + 问答 + 求 职交流为一体，是一个综合类的具身社区，近2000人了。我们期望未来2年内做到近万人的规模。给 大家打造一个交流+技术分享的聚集地，是许多初学者和进阶的同学经常逛的地方。 社区内部经常为大家解答各类实用问题：如何使用设备？如何有效采集数据？如何部署VA、VLA模 型等。是采集背景太复杂还是数据比较dirty? 快速解答，方便大家应用到项目中。 一个社区能在大家最需要帮助的时候解决问题，无疑是非常有价值的。具身智能之心知识星球（国 内首个具身全栈技术社区），目前已经完成了产业、学术、求职、问答交流等多个领域的闭环。遇 到什么问题就分享什么解决方案，哪块研究最前沿，就给大家源源不断提供解决思路，还有求职 ...

Group 1 - The article announces the recruitment of teachers for embodied intelligence training, targeting both B-end (business) and C-end (consumer) training services, with compensation above industry standards [1] - The training covers various advanced topics including VLA, VLN, remote operation, Diffusion Policy, reinforcement learning, sim2real, multimodal large models, simulation, motion control, and target navigation [2] - B-end training is aimed at enterprises, universities, and research institutions, while C-end training focuses on students and job seekers, with responsibilities including curriculum design and material preparation [3] Group 2 - Candidates are required to have a doctoral degree or higher (including those currently enrolled), with a preference for those who have published two papers in A-level or Q1 journals/conferences, or have two years of industry experience [3] - Interested individuals can add a specified WeChat contact for further inquiries [4]

Core Viewpoint - The article discusses the EgoTwin framework, which allows for the simultaneous generation of first-person perspective videos and human actions, achieving precise alignment in both time and space, thus attracting significant attention in the AR/VR, embodied intelligence, and wearable device sectors [2][5]. Summary by Sections Introduction - The EgoTwin framework was developed collaboratively by institutions including the National University of Singapore, Nanyang Technological University, Hong Kong University of Science and Technology, and Shanghai Artificial Intelligence Laboratory [2]. Key Highlights - EgoTwin combines first-person perspective video generation with human action generation, addressing challenges such as camera trajectory alignment with head movement and establishing a causal loop between observation and action [8]. - It utilizes a large dataset of 170,000 segments of first-person multimodal real-world scenes for training, leading to significant performance improvements, including a 48% reduction in trajectory error and a 125% increase in hand visibility F-score [8]. Method Innovations - EgoTwin introduces three core technologies: 1. A head-centric action representation that directly provides head 6D pose, reducing alignment errors [12]. 2. A bidirectional causal attention mechanism that allows for causal interactions between action tokens and video tokens [12]. 3. An asynchronous diffusion mechanism that ensures synchronization while allowing independent noise addition and removal on different timelines [12]. Technical Implementation - The model employs a three-channel diffusion architecture, optimizing computational efficiency by reusing only the necessary layers for the action branch [13]. - The training process involves three phases, including initial training of the action VAE, followed by alignment training, and finally joint fine-tuning of all three modalities [21]. Data and Evaluation - EgoTwin is trained and tested on the Nymeria dataset, which includes 170,000 five-second video clips covering various daily actions [17]. - A comprehensive evaluation system is established to measure generation quality and consistency across the three modalities, utilizing metrics such as I-FID, FVD, and CLIP-SIM [17]. Quantitative Experiments - EgoTwin outperforms baseline methods across all nine evaluation metrics, demonstrating significant improvements in trajectory alignment and hand score, while also enhancing the fidelity and consistency of generated videos and actions [18][19]. Generation Modes - The framework supports three generation modes: T2VM (text to video and motion), TM2V (text and motion to video), and TV2M (text and video to motion), showcasing its versatility [24].