Core Viewpoint - The article discusses the advancements in autonomous driving technology, particularly focusing on the MindVLA framework, which integrates spatial intelligence, linguistic intelligence, action policy, and reinforcement learning to enhance vehicle autonomy and interaction capabilities. Group 1: MindVLA Framework Overview - MindVLA consists of four main modules: spatial intelligence, linguistic intelligence, action policy, and reinforcement learning, each serving distinct functions in the autonomous driving process [5][6]. - The spatial intelligence module utilizes multi-modal sensor data and a 3D encoder to extract spatiotemporal features, merging sensor and semantic information into a unified representation [5]. - The linguistic intelligence module employs a large language model (MindGP) for joint reasoning between spatial and language inputs, facilitating human-vehicle interaction through voice commands [5]. - The action policy module generates future vehicle behavior trajectories using diffusion models, introducing noise to guide the generation process for diverse action planning [5]. - The reinforcement learning module simulates external environment responses to evaluate actions and optimize behavior through continuous learning [5]. Group 2: GaussianAD Framework - The GaussianAD framework addresses the limitations of traditional end-to-end autonomous driving by using Gaussian representations for 3D scene initialization and interaction [12][10]. - It employs a 4D sparse convolution approach to extract multi-scale features from panoramic images, optimizing Gaussian parameters to create a sparse 3D semantic Gaussian set [16][12]. - The advantages of Gaussian representation include reduced computational redundancy while maintaining fine-grained 3D structure, significantly enhancing downstream task performance [16][15]. Group 3: Linguistic Intelligence Module - The linguistic intelligence module is designed to create a customized large language model (LLM) that is specifically trained on relevant data for autonomous driving, enhancing its spatial reasoning and language capabilities [18][19]. - The model architecture incorporates sparse design to improve inference performance while reducing capacity [18]. Group 4: Action Policy and Trajectory Generation - The action policy utilizes a diffusion model to decode action tokens into trajectories, enhancing the model's ability to navigate complex traffic environments [22][24]. - TrajHF, a component of the action policy, generates diverse trajectories through multi-conditional denoising and reinforcement learning fine-tuning, aligning generated trajectories with human driving preferences [25][26]. - The model structure includes a generative trajectory model and reinforcement learning fine-tuning to maximize human preference rewards, addressing the challenges of traditional imitation learning [28][30]. Group 5: Preference Data Construction - The process of constructing preference data involves labeling driving data with different driving style tags, focusing on key frames where significant actions occur [31][33]. - The key frame annotation process is designed to ensure data quality through random manual checks, allowing for large-scale annotation of driving preferences [31][33].