人形机器人的真机强化学习! ICLR 2026 通研院提出人形机器人预训练与真机微调新范式

Core Insights - The article discusses the advancements in humanoid robots, particularly their ability to perform complex tasks like dancing and running, while emphasizing the importance of continuous reinforcement learning in real-world environments [2][3] - The LIFT framework proposed by researchers aims to bridge the gap between large-scale pretraining and efficient fine-tuning for humanoid control, addressing the limitations of existing methods [9][12] Group 1: Background and Motivation - Current humanoid robots primarily rely on on-policy algorithms like PPO for pretraining, which are not effective for continuous learning due to safety and economic concerns [7] - The main challenge is to achieve large-scale pretraining speed without sacrificing sample efficiency and safety during the fine-tuning phase [9] Group 2: LIFT Framework - LIFT utilizes off-policy reinforcement learning algorithms like SAC for large-scale pretraining, which allows for better sample efficiency when data is limited [12][15] - The framework incorporates a physics-informed world model to enhance prediction performance and fine-tuning efficiency [12][18] Group 3: Experimental Results - LIFT demonstrated significant advantages over baseline methods like PPO and SAC in terms of convergence time and sample efficiency during pretraining and fine-tuning [20][24] - The framework allows for zero-shot deployment of pre-trained policies to real-world robots, showcasing its effectiveness in real-time applications [20][22] Group 4: Challenges and Future Directions - The article highlights several bottlenecks that need to be addressed for scaling reinforcement learning in real-world applications, including observation and state estimation, safety mechanisms, and system throughput [41]