Core Viewpoint - The integration of Vision-Language-Action (VLA) models with Reinforcement Learning (RL) presents a promising new paradigm that leverages both environmental trial-and-error interactions and pre-collected suboptimal data for enhanced performance [2]. Group 1: Offline RL Training without Environment - The paper "MoRE: Unlocking Scalability in Reinforcement Learning for Quadruped Vision-Language-Action Models" discusses scalability in RL applications [3]. - "Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions" focuses on offline RL techniques [3]. Group 2: Online RL Training with Environment - Online RL training enhances VLA models through trial-and-error interactions in real-time environments, leading to performance improvements [4]. - The paper "ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning" explores this concept [5]. - "GeRM: A Generalist Robotic Model with Mixture-of-experts for Quadruped Robot" presents a generalist approach in robotic models [5]. Group 3: Simulator-Based Approaches - Various projects aim to improve VLA models using simulation environments, such as "OctoNav: Towards Generalist Embodied Navigation" [6]. - "TGRPO: Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization" focuses on optimizing VLA models through trajectory-based methods [6]. - "VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning" emphasizes scalable RL for robotic manipulation [6]. Group 4: Real-World Applications - The deployment phase of RL training is crucial for testing VLA models in real-world scenarios [8]. - "Dynamism v1 (DYNA-1) Model: A Breakthrough in Performance and Production-Ready Embodied AI" highlights advancements in embodied AI [9]. - "ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy" discusses fine-tuning methods for VLA models [9]. Group 5: RL Alignment Training - "GRAPE: Generalizing Robot Policy via Preference Alignment" addresses the alignment of robot policies with user preferences [11]. - "SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning" focuses on safety in VLA model training [12].