Workflow
Sim2Real
icon
Search documents
清华团队开源DISCOVERSE框架:用3D高斯渲染打通机器人仿真到现实的“最后一公里”!
机器人大讲堂· 2025-11-10 04:07
Core Insights - The article discusses the challenges in end-to-end robot learning, particularly focusing on the "Sim2Real" gap, which is primarily caused by the inadequacy of simulation environments to accurately replicate real-world scenarios [1][6][10]. Group 1: Challenges in Robot Simulation - Current simulation environments struggle with three main issues: insufficient realism in replicating real-world scenarios, high costs in scene asset acquisition and system configuration, and time-consuming data collection processes [1][5]. - The core obstacle is the performance drop during the Sim2Real transfer, which stems from the fundamental differences between simulated and real-world environments, such as object appearance, lighting effects, and spatial geometry [1][6]. Group 2: Existing Simulation Frameworks - Various simulation frameworks have been developed, but none meet the three critical requirements: high visual fidelity, accurate physical interaction, and efficient parallel scalability [3][6]. - Traditional simulators often compromise on either visual realism or physical accuracy, leading to ineffective training for robots [6][7]. Group 3: DISCOVERSE Framework - DISCOVERSE is an open-source simulation framework developed by Tsinghua University in collaboration with other institutions, integrating 3D Gaussian Splatting (3DGS), MuJoCo physics engine, and control interfaces into a unified architecture [5][10]. - The framework aims to bridge the Sim2Real gap by enhancing the realism of simulations through a three-layer innovation approach, focusing on accurate digital representation of real-world scenes and objects [10][12]. Group 4: Performance and Efficiency - DISCOVERSE significantly improves simulation speed, achieving rendering speeds up to 650 FPS on high-end hardware, which is three times faster than competing solutions [19][20]. - The framework supports a wide range of asset formats and robot models, enhancing compatibility and reducing the need for extensive configuration [21][22]. Group 5: Testing and Results - In comparative tests, DISCOVERSE outperformed other mainstream simulators in zero-shot transfer success rates across various tasks, demonstrating its effectiveness in real-world applications [24][27]. - The framework also enhances data collection efficiency, reducing the time required to gather demonstration data from 146 minutes to just 1.5 minutes, thus accelerating algorithm iteration [29]. Group 6: Future Implications - DISCOVERSE is positioned as a versatile robot simulation framework capable of supporting various complex tasks, with potential applications in robotics, drones, and autonomous driving sensors [30]. - The release of the framework's code and API aims to facilitate adoption by developers and enterprises, marking a significant milestone in the robotics industry [30].
Richtech Robotics Offers First Look at Dex: A Mobile Humanoid Robot for Real-World Work
Globenewswire· 2025-10-28 18:30
Core Insights - Richtech Robotics has launched Dex, its first mobile humanoid robot designed for industrial applications, in collaboration with NVIDIA to enhance its capabilities [1][3] - Dex utilizes NVIDIA Jetson Thor technology for real-time reasoning and complex task execution, operating efficiently for a full workday on a single charge [2][5] - The robot combines insights from over 450 previous deployments, integrating autonomous mobile robot (AMR) technology with dual-armed precision to enhance operational efficiency [4][5] Technology and Development - Richtech employs NVIDIA's Isaac Sim for training Dex in diverse industrial contexts, facilitating a "Sim2Real" pipeline that accelerates deployment and improves safety [3][6] - The robot's design prioritizes mobility and dexterity, featuring a wheeled platform for stability and lower energy costs, while maintaining a four-hour battery life in mobile mode [4][5] - Richtech is launching an American robotics data initiative to collect regionally grounded data, aiming to empower the development of physical AI in the U.S. [7] Applications and Capabilities - Dex is capable of performing a variety of light to medium industrial tasks, making it a valuable asset in manufacturing and logistics sectors [8][9] - The robot's features include modular end-effectors for various tools, a four-camera vision system for navigation, and the ability to operate continuously from a static base [5][6] - Richtech invites companies to explore pilot opportunities with Dex, showcasing its capabilities at industry events [10][9] Company Overview - Richtech Robotics focuses on developing advanced robotic solutions and data infrastructure, emphasizing automation and continuous AI-driven improvement across various sectors [11]
机械设备行业专题研究:机器人大脑是商业化焦点,Sim2real或成主流训练方案
GOLDEN SUN SECURITIES· 2025-10-26 09:06
Investment Rating - The report maintains an "Accumulate" rating for the mechanical equipment industry [4]. Core Insights - The focus of commercialization is on robotic brains, with Sim2Real potentially becoming the mainstream training method [2]. - The development of humanoid robot models is rapidly advancing, with Tesla's Optimus model demonstrating a high degree of human-like capabilities [3][29]. - The report suggests paying attention to listed companies involved in related hardware and software businesses, such as Pinming Technology [3]. Summary by Sections Section 1: Robotic Brain Development - The evolution from LLM to VLM and then to VLA models is enhancing the generalization and precision of robotic actions [1]. - VLA models are increasingly incorporating tactile inputs to improve robustness [1]. Section 2: Sim2Real Technology - Sim2Real utilizes synthetic data generation to help robots accumulate experience through diverse scenarios, linking virtual and real-world data for training [2]. - The technology involves a tri-computer setup: an AI supercomputer, a simulation computer, and a physical AI computer [2]. Section 3: Tesla's Optimus Model - Tesla's Optimus integrates AI systems from FSD and xAI's Grok model, achieving high levels of human-like interaction and physical self-awareness [3][29]. - The model's architecture allows it to process various sensory data to generate action commands directly from raw sensor inputs [33]. Section 4: Emerging Technologies and Models - The report discusses several innovative models, including RT-1, RT-2, Magma, and ViLLA, each contributing to bridging the gap between visual/textual inputs and robotic actions [14][17][22]. - The introduction of the "force-position hybrid control algorithm" by the Beijing General Artificial Intelligence Research Institute shows significant improvements in task success rates [58]. Section 5: Market Trends - The mechanical equipment industry is projected to experience varying growth rates, with a notable increase expected in the coming years [5].
黄仁勋女儿首秀直播:英伟达具身智能布局藏哪些关键信号?
机器人大讲堂· 2025-10-15 15:32
Core Insights - The discussion focuses on bridging the Sim2Real gap in robotics, emphasizing the importance of simulation in training robots to operate effectively in the real world [2][4][10] Group 1: Key Participants and Context - Madison Huang, NVIDIA's head of Omniverse and physical AI marketing, made her first public appearance in a podcast discussing robotics and simulation [1][2] - The conversation featured Dr. Xie Chen, CEO of Lightwheel Intelligence, who has extensive experience in the Sim2Real field, having previously led NVIDIA's autonomous driving simulation efforts [2][9] Group 2: Challenges in Robotics - The main challenges in bridging the Sim2Real gap are identified as perception differences, physical interaction discrepancies, and scene complexity variations [4][6] - Jim Fan, NVIDIA's chief scientist, highlighted that generative AI technologies could enhance the realism of simulations, thereby reducing perception gaps [6][7] Group 3: Importance of Simulation - Madison Huang stated that robots must experience the world rather than just read data, as real-world data collection is costly and inefficient [7][9] - The need for synthetic data is emphasized, as it can provide a scalable solution to the data scarcity problem in robotics [9][10] Group 4: NVIDIA's Technological Framework - NVIDIA's approach involves a "three-computer" logic: an AI supercomputer for processing information, a simulation computer for training in virtual environments, and a physical AI computer for real-world task execution [10][11] - The simulation computer, powered by Omniverse and Isaac Sim, is crucial for developing robots' perception and interaction capabilities [11][12] Group 5: Collaboration with Lightwheel Intelligence - The partnership with Lightwheel Intelligence is highlighted as essential for NVIDIA's physical AI ecosystem, focusing on solving data bottlenecks in robotics [15][16] - Both companies share a vision for SimReady assets, which must possess real physical properties to enhance simulation accuracy [16][15] Group 6: Future Directions - The live discussion is seen as an informal introduction to NVIDIA's physical intelligence strategy, which aims to create a comprehensive ecosystem for robotics [18] - As collaboration deepens, it is expected to transform traditional robotics technology pathways [18]
直播预告:光轮智能 × NVIDIA带来Sim2Real关键突破
量子位· 2025-10-08 13:06
Core Viewpoint - The collaboration between Guanglun Intelligent and NVIDIA aims to leverage SimReady and AI to achieve seamless migration from virtual simulation to the physical world, addressing key challenges in robot development and implementation [2][3]. Group 1: Live Broadcast Highlights - The live broadcast will focus on the technological breakthrough of Sim2Real, detailing how both companies utilize SimReady and AI to overcome challenges in robot development [2]. - Experts will share insights on the technological trends and commercialization paths in the fields of robotics and AI, drawing from their practical experiences [4]. Group 2: Collaboration Progress - Exclusive updates on the latest achievements and plans in technology research and application scenarios from the partnership between Guanglun Intelligent and NVIDIA will be disclosed [3]. Group 3: Key Speakers and Event Details - The live broadcast will feature Steve Xie, the founder and CEO of Guanglun Intelligent, and Madison Huang, Senior Director of Product Marketing at NVIDIA [6]. - The event is scheduled for October 9 at 00:00 Beijing time, which corresponds to October 8 at 09:00 Pacific time [6].
Sim2Real,解不了具身智能的数据困境。
自动驾驶之心· 2025-10-03 03:32
Core Viewpoint - The article discusses the ongoing debate in the field of embodied intelligence regarding the reliance on simulation efficiency versus real-world data, and the potential of world models to redefine the landscape of data utilization in this domain [4][8]. Group 1: Understanding Sim-to-Real Gap - The "Sim-to-Real gap" refers to the discrepancies between simulated environments and real-world scenarios, primarily due to incomplete simulations that fail to accurately replicate visual and physical details [8]. - Research indicates that the gap exists because simulation models do not fully capture the complexities of the real world, leading to limited generalization capabilities and a focus on specific scenarios [8][11]. - Solutions to bridge this gap involve optimizing data, including designing virtual and real data ratios and leveraging AIGC to generate diverse datasets that balance volume and authenticity [11][12]. Group 2: Data Utilization in Embodied Intelligence - There is a consensus among experts that while real data is ideal for training, the current landscape necessitates a reliance on simulation data due to the scarcity of high-quality real-world datasets in the embodied intelligence field [20][21]. - Simulation data plays a crucial role in foundational model iteration and testing, as it allows for safe and efficient algorithm testing before deploying on real machines [21][24]. - The potential of simulation in scaling reinforcement learning is highlighted, as well-constructed simulators can facilitate large-scale parallel training, enabling models to learn from scenarios that are difficult to capture in real life [24][26]. Group 3: World Models and Future Directions - The article emphasizes the significance of world models in future research, particularly in areas like autonomous driving and embodied intelligence, showcasing their potential in general visual understanding and long-term planning [30][32]. - Challenges remain in automating the generation of simulation data and ensuring the diversity and generalization of actions within simulations, which are critical for advancing the field [28][29]. - The introduction of new modalities, such as force and touch, into world models is suggested as a promising direction for future research, despite current limitations in computational resources [30][31]. Group 4: Reaction to Boston Dynamics Technology - Experts acknowledge the advanced capabilities of Boston Dynamics robots, particularly their smooth execution of complex tasks that require sophisticated motion control [33][37]. - The discussion highlights the importance of hardware and data in the field of embodied intelligence, with Boston Dynamics' approach serving as a benchmark for future developments [37][39]. - The consensus is that the seamless performance of these robots is attributed not only to hardware differences but also to superior motion control techniques that could inform future research in embodied intelligence [39][41].
自搜索强化学习SSRL:Agentic RL的Sim2Real时刻
机器之心· 2025-09-02 01:27
Core Insights - The article discusses the development and effectiveness of SSRL (Structured Search Reinforcement Learning) in enhancing the training efficiency and stability of Search Agents using large language models (LLMs) [6][28] - SSRL demonstrates superior performance over traditional methods that rely on external search engines, achieving effective transfer from simulation to real-world applications (Sim2Real) [6][28] Group 1 - SSRL utilizes structured prompts and format rewards to effectively extract world knowledge from models, leading to improved performance across various benchmarks and reduced hallucination [2][6] - The research highlights the high costs and inefficiencies associated with current RL training methods for Search Agents, which include full-real and semi-real search approaches [7][13] - The introduction of SSRL allows for a significant increase in training efficiency, estimated at approximately 5.6 times, while maintaining a continuous increase in training rewards without collapse [31][32] Group 2 - Experiments show that models trained with SSRL outperform those relying on external engines, particularly in real-world search scenarios, indicating the importance of integrating real-world knowledge [28][31] - The article presents findings that suggest the combination of self-generated knowledge and real-world knowledge can enhance model performance, particularly through entropy-guided search strategies [34] - The integration of SSRL with TTRL (Task-Driven Reinforcement Learning) has shown to improve generalization and effectiveness, achieving up to a 67% performance increase in certain tasks [38][39]
数据困局下的具身智能,谁能率先破局?
机器之心· 2025-08-10 01:30
Group 1 - The core issue in embodied intelligence is the severe shortage of real data, with most robotic models relying on less than 1% of real operational data, which limits their generalization capabilities in complex environments [5][6] - There is a debate in the industry regarding the importance of real data versus synthetic simulation data, which affects the scalability and generalization of embodied intelligence [6][7] - Some experts argue that while synthetic data has advantages in cost and scalability, it cannot fully replicate the complexities of the real world, leading to a "domain gap" that hinders model transferability [7][8] Group 2 - The need for hundreds of billions of real data points is highlighted, with current datasets only reaching the million level, presenting a significant bottleneck for the development of embodied intelligence [8] - The strategy of using synthetic data for initial training followed by fine-tuning with real data is seen as a key pathway for the cold start and scaling of embodied intelligence [8][9] - Teleoperation is emerging as a primary method for acquiring real data, especially in the early stages of embodied intelligence, where human operators provide high-quality demonstration actions for training [9][10]
科协年会助力青年人才挑大梁
Ke Ji Ri Bao· 2025-08-03 03:43
Core Insights - The 27th Annual Conference of the China Association for Science and Technology (CAST) was held in Beijing from July 1 to 31, focusing on "Tracing Technological Frontiers to Support Innovative Development" [1] - The conference attracted over 7,000 participants, including more than 110 academicians, with 57% of attendees being young scientists under 40 years old [1] - A total of over 990 high-level academic reports were presented during the conference [1] Group 1 - One highlight of the conference was the deep involvement and leadership of young scientists in frontier discussions [2] - The conference fostered an atmosphere of equal communication, allowing young scholars to directly question academicians and experts, which is beneficial for breaking cognitive biases and enhancing problem understanding [4] - The design of forums encouraged embracing uncertainty in research and promoted non-consensus viewpoints, creating a more inclusive and open academic environment [4] Group 2 - Participants engaged in discussions on cutting-edge topics such as "Sim2Real challenges" and "multimodal perception fusion" in the "Embodied Intelligent Robots" forum, inspiring new research directions [3] - The "Key Technologies for Commercialization of Controlled Nuclear Fusion" forum attracted diverse participants from academia, industry, and technology sectors, facilitating in-depth discussions on uncertain topics related to nuclear fusion [4] - The conference emphasized the importance of fostering divergent thinking and academic innovation through collaborative discussions among participants of varying expertise and age [4]
顶尖科学家带队,国内头部具身智能机器人企业完成数亿元新一轮融资!
Robot猎场备忘录· 2025-07-20 05:01
Core Viewpoint - Kuawei Intelligent, a leading domestic embodied intelligent robot company, has completed several hundred million yuan in A1 and A2 round financing, with plans to increase investment in technology research and product innovation [1][3]. Financing History - On July 10, 2025, Kuawei Intelligent completed A1 and A2 round financing, led by Chengdu Kechuang Investment and Hongtai Fund, with participation from various well-known institutions [2]. - The company had previously completed a Pre-A+ round financing on January 16, 2025, with undisclosed amounts [2]. - Kuawei Intelligent has completed a total of eight financing rounds, including angel and Pre-A rounds, with significant investments from various venture capital firms [2]. Company Overview - Kuawei Intelligent was established on June 15, 2021, in Shenzhen, initially focusing on 3D vision software and hardware product development [4]. - The company has evolved to focus on developing highly versatile embodied intelligent technology, positioning itself as a national high-tech enterprise [5]. Product Launch - On January 20, 2025, Kuawei Intelligent launched the DexForce W1, the industry's first humanoid robot based on the Sim2Real embodied intelligent engine [8][9]. - The W1 robot features a height of 170 cm, a maximum arm load of 20 kg, and an 8-hour battery life, with capabilities for autonomous charging and folding [11]. Technological Advancements - The DexVerse embodied intelligent engine enables rapid deployment and application switching in various environments, enhancing the robot's operational capabilities [11][12]. - Kuawei Intelligent has developed a comprehensive product matrix that includes the embodied intelligent brain, sensors, and general-purpose robots [12][14]. Commercialization Strategy - Initially focused on industrial robots, Kuawei Intelligent has shifted its focus to embodied large models and simulation data, emphasizing the end-to-end capabilities of the DexVerse engine [16]. - The company has achieved significant commercial success, with annual revenue reaching the billion yuan level and serving over a hundred clients across various industries [16]. Industry Context - The humanoid robot sector is divided into two main camps: hardware-focused companies like Yushu Technology and software-driven companies like ZhiYuan Robotics [17]. - Kuawei Intelligent is positioned in the software camp, emphasizing the importance of strong AI capabilities for commercialization in the humanoid robot market [17][19].