Core Viewpoint - The article discusses the advancements and potential of reinforcement learning (RL) in the field of autonomous driving, highlighting its evolution and comparison with other learning paradigms such as supervised learning and imitation learning [4][7][8]. Summary by Sections Background - The article notes the recent industry focus on new technological paradigms like VLA and reinforcement learning, emphasizing the growing interest in RL following significant milestones in AI, such as AlphaZero and ChatGPT [4]. Supervised Learning - In autonomous driving, perception tasks like object detection are framed as supervised learning tasks, where a model is trained to map inputs to outputs using labeled data [5]. Imitation Learning - Imitation learning involves training models to replicate actions based on observed behaviors, akin to how a child learns from adults. This is a primary learning objective in end-to-end autonomous driving [6]. Reinforcement Learning - Reinforcement learning differs from imitation learning by focusing on learning through interaction with the environment, using feedback from task outcomes to optimize the model. It is particularly relevant for sequential decision-making tasks in autonomous driving [7]. Inverse Reinforcement Learning - Inverse reinforcement learning addresses the challenge of defining reward functions in complex tasks by learning from user feedback to create a reward model, which can then guide the main model's training [8]. Basic Concepts of Reinforcement Learning - Key concepts include policies, rewards, and value functions, which are essential for understanding how RL operates in autonomous driving contexts [14][15][16]. Markov Decision Process - The article explains the Markov decision process as a framework for modeling sequential tasks, which is applicable to various autonomous driving scenarios [10]. Common Algorithms - Various algorithms are discussed, including dynamic programming, Monte Carlo methods, and temporal difference learning, which are foundational to reinforcement learning [26][30]. Policy Optimization - The article differentiates between on-policy and off-policy algorithms, highlighting their respective advantages and challenges in training stability and data utilization [27][28]. Advanced Reinforcement Learning Techniques - Techniques such as DQN, TRPO, and PPO are introduced, showcasing their roles in enhancing training stability and efficiency in reinforcement learning applications [41][55]. Application in Autonomous Driving - The article emphasizes the importance of reward design and closed-loop training in autonomous driving, where the vehicle's actions influence the environment, necessitating sophisticated modeling techniques [60][61]. Conclusion - The rapid development of reinforcement learning algorithms and their application in autonomous driving is underscored, encouraging practical engagement with the technology [62].

Core Insights - The article presents a novel approach to trajectory prediction in autonomous driving, emphasizing a "First Reasoning, Then Forecasting" strategy that integrates intention reasoning to enhance prediction accuracy and reliability [2][4][48]. Group 1: Methodology - The proposed method introduces an intention reasoner based on a query-centric Inverse Reinforcement Learning (IRL) framework, which captures the behavior of traffic participants and their intentions in a compact representation [2][6][48]. - A bidirectional selective state space model (Bi-Mamba) is developed to improve trajectory decoding, effectively capturing the sequential dependencies of trajectory states [7][9][48]. - The framework utilizes a grid-level graph to represent the driving context, allowing for efficient modeling of participant behavior and intentions [5][6][20]. Group 2: Experimental Results - Extensive experiments on large datasets such as Argoverse and nuScenes demonstrate that the proposed method significantly enhances prediction confidence and achieves competitive performance compared to state-of-the-art models [9][34][38]. - In the Argoverse 1 dataset, the proposed method (FiM) outperformed several strong baseline methods in key metrics such as Brier score and minFDE6, indicating its robust predictive capabilities [34][35]. - The results from Argoverse 2 further validate the effectiveness of the intention reasoning strategy, showing that longer-term intention supervision improves prediction reliability [36][37]. Group 3: Challenges and Innovations - The article highlights the inherent challenges in modeling intentions due to the complexity of driving scenarios, advocating for the use of large reasoning models (LRMs) to enhance intention inference [5][6][12]. - The integration of a dense occupancy grid map (OGM) prediction head is introduced to model future interactions among participants, which enhances the overall prediction performance [7][25][41]. - The study emphasizes the importance of intention reasoning in motion prediction, establishing a promising baseline for future research in trajectory prediction [48].

Core Viewpoint - The article discusses the advancements and applications of reinforcement learning (RL) in the field of autonomous driving, highlighting its potential to enhance decision-making processes in dynamic environments. Group 1: Background and Concepts - The concept of VLA (Variational Learning Algorithm) and its relation to embodied intelligence is introduced, emphasizing its similarity to end-to-end autonomous driving [3] - Reinforcement learning has gained traction in various industries following significant milestones like AlphaZero in 2018 and ChatGPT in 2023, showcasing its broader applicability [3] - The article aims to explain reinforcement learning from a computer vision perspective, drawing parallels with established concepts in the field [3] Group 2: Learning Methods - Supervised learning in autonomous driving involves tasks like object detection, where a model is trained to map inputs to outputs using labeled data [5] - Imitation learning is described as a method where models learn actions by mimicking human behavior, akin to how children learn from adults [6] - Reinforcement learning differs from imitation learning by focusing on optimizing actions based on feedback from interactions with the environment, making it suitable for sequential decision-making tasks [7] Group 3: Advanced Learning Techniques - Inverse reinforcement learning is introduced as a method to derive reward functions from expert data, particularly useful when defining rewards is challenging [8] - The Markov Decision Process (MDP) is explained as a framework for modeling decision-making tasks, where states, actions, and rewards are interrelated [9] - Dynamic programming and Monte Carlo methods are discussed as techniques for solving reinforcement learning problems, emphasizing their role in optimizing decision-making processes [11][12] Group 4: Reinforcement Learning Algorithms - Various reinforcement learning algorithms are categorized, including on-policy and off-policy methods, highlighting their differences in training stability and data utilization [25][26] - The article outlines key algorithms such as Q-learning, SARSA, and policy gradient methods, explaining their mechanisms and applications in reinforcement learning [27][29] - Advanced algorithms like TRPO and PPO are presented, focusing on their strategies for ensuring stable training and optimizing policy updates [57][58] Group 5: Applications in Autonomous Driving - The importance of reward design in autonomous driving is emphasized, with safety, comfort, and efficiency being key factors [62] - The article discusses the need for closed-loop training systems in autonomous driving, where vehicle actions influence the environment, necessitating dynamic modeling of other vehicles [62] - The integration of end-to-end learning with reinforcement learning is highlighted as a method to adapt to changing environments in real-time [63]

预测行驶中的交通参与者的轨迹运动，对于确保自动驾驶系统的安全性而言，既是一项重大挑战，也是一 项至关重要的需求。与大多数现有的、直接预测未来轨迹的数据驱动方法不同，我们从 规划（planning） 的视角 重新思考这一任务，提出一种" 先推理，后预测（First Reasoning, Then Forecasting） "的策略，该 策略显式地将行为意图作为轨迹预测的空间引导。为实现这一目标，进一步引入了一种可解释的、基于奖 励的意图推理器（intention reasoner），其建立在一种新颖的 以查询为中心的逆强化学习（query-centric Inverse Reinforcement Learning, IRL） 框架之上。我们的方法首先将交通参与者和场景元素编码为统一的 向量化表示，然后通过以查询为中心的范式聚合上下文特征。进而推导出一个 奖励分布（reward distribution） ——一种紧凑但信息丰富的表示，用于刻画目标参与者在给定场景上下文中的行为。在该奖 励启发式（reward heuristic）的引导下，我们进行策略 rollout，以推理多种可能的意图，从而为后续的轨迹 生 ...