最近很多同学咨询柱哥端到端和VLA的方案，相比于模块化方法，统一的感知、规控建模会带来更高的智驾能力上限，这些方案的技术难度上也更大。今天自动 驾驶之心汇总了行业里面参考最多的端到端和VLA算法。 更多内容欢迎移步自动驾驶之心知识星球，一个交流技术和方案的地方。 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近15个 方向 学习 路线 端到端思维导图一览。整体来说，端到端分为一段式和二段式两个大的方向，二段式更偏联合预测一些，感知作为输入，模型聚焦于自车的轨迹规划和他车的轨 迹预测。而一段式端到端则是建模从传感器输入到自车轨迹的输出的过程。具体又可以细分为基于感知的一段式端到端（UniAD）、基于扩散模型的一段式端到 端（DiffusionDrive）、基于世界模型的一段式端到端（Drive-OccWorld），这些方法各有侧重点，量产中会结合各种方法的优点进行模型优化。 而VLA则是VLM+E2E的延伸，期望通过大模型的能力赋予量产模型更高的场景理解能力，目前星球内部梳理了语言模型作为解释器的相关工作、模块化VLA汇 总、统一端到端VLA梳理及推理增强VLA的诸多算法汇总。星友们 ...

Core Viewpoint - The article emphasizes the importance of creating an engaging learning environment in the field of autonomous driving and AI, aiming to bridge the gap between industry and academia while providing valuable resources for students and professionals [1]. Group 1: Community and Resources - The community has established a closed loop across various fields including industry, academia, job seeking, and Q&A exchanges, focusing on what type of community is needed [1][2]. - The platform offers cutting-edge academic content, industry roundtables, open-source code solutions, and timely job information, streamlining the search for resources [2][3]. - A comprehensive technical roadmap with over 40 technical routes has been organized, catering to various interests from consulting applications to the latest VLA benchmarks [2][14]. Group 2: Educational Content - The community provides a series of original live courses and video tutorials covering topics such as automatic labeling, data processing, and simulation engineering [4][10]. - Various learning paths are available for beginners, as well as advanced resources for those already engaged in research, ensuring a supportive environment for all levels [8][10]. - The community has compiled a wealth of open-source projects and datasets related to autonomous driving, facilitating quick access to essential materials [25][27]. Group 3: Job Opportunities and Networking - The platform has established a job referral mechanism with multiple autonomous driving companies, allowing members to submit their resumes directly to desired employers [4][11]. - Continuous job sharing and position updates are provided, contributing to a complete ecosystem for autonomous driving professionals [11][14]. - Members can freely ask questions regarding career choices and research directions, receiving guidance from industry experts [75]. Group 4: Technical Focus Areas - The community covers a wide range of technical focus areas including perception, simulation, planning, and control, with detailed learning routes for each [15][29]. - Specific topics such as 3D target detection, BEV perception, and online high-precision mapping are thoroughly organized, reflecting current industry trends and research hotspots [42][48]. - The platform also addresses emerging technologies like visual language models (VLM) and diffusion models, providing insights into their applications in autonomous driving [35][40].

写在前面 & 笔者的个人理解 扩散模型在自动驾驶场景视频生成中已经展现出比较有前景的视觉生成质量。然而，现有的基于视频扩散的世界模型在灵活长度、长时序预测以及轨迹规划方面 仍存在不足。这是因为传统视频扩散模型依赖于对固定长度帧序列的全局联合分布建模，而非逐步构建每个时间下的局部化分布。本研究提出 Epona ，一种自回 归扩散世界模型，通过两项关键创新实现局部时空分布建模：1） 解耦的时空分解 ，将时间动态建模与细粒度未来世界生成分离；2） 模块化的轨迹与视频预测 ，通过端到端框架无缝整合运动规划与视觉建模。本文的架构通过引入一种新的"链式前向训练策略"（chain-of-forward training strategy），在实现高分辨率、长持 续时间生成的同时解决了自回归循环中的误差累积问题。实验结果表明，与现有方法相比，Epona在FVD指标上提升7.4%，预测时长可达数分钟。该世界模型进一 步可作为实时端到端规划器，在NAVSIM基准测试中优于现有端到端规划器。 点击下方 卡片 ，关注" 自动驾驶之心 "公众号 戳我-> 领取 自动驾驶近15个 方向 学习 路线 今天自动驾驶之心为大家分享 地平线联合 ...

Core Viewpoint - GeoDrive is a next-generation world model system for autonomous driving, developed collaboratively by Peking University, Berkeley AI Research (BAIR), and Li Auto, addressing the limitations of existing methods that rely on 2D modeling and lack 3D spatial perception, which can lead to unreasonable trajectories and distorted dynamic interactions [11][14]. Group 1: Key Innovations - **Geometric Condition-Driven Generation**: Utilizes 3D rendering to replace numerical control signals, effectively solving the action drift problem [6]. - **Dynamic Editing Mechanism**: Injects controllable motion into static point clouds, balancing efficiency and flexibility [7]. - **Minimized Training Cost**: Freezes the backbone model and employs lightweight adapters for efficient data training [8]. - **Pioneering Applications**: Achieves real-time scene editing and VLA (Vision-Language-Action) collaborative planning within the driving world model for the first time [9][10]. Group 2: Technical Details - **3D Geometry Integration**: The system constructs a 3D representation from single RGB images, ensuring spatial consistency and coherence in scene structure [12][18]. - **Dynamic Editing Module**: Enhances the realism of multi-vehicle interaction scenarios during training by allowing flexible adjustments of movable objects [12]. - **Video Diffusion Architecture**: Combines rendered conditional sequences with noise features to enhance 3D geometric fidelity while maintaining photorealistic quality [12][33]. Group 3: Performance Metrics - GeoDrive significantly improves controllability of driving world models, reducing trajectory tracking error by 42% compared to the Vista model, and shows superior performance across various video quality metrics [19][34]. - The model demonstrates effective generalization to new perspective synthesis tasks, outperforming existing models like StreetGaussian in video quality [19][38]. Group 4: Conclusion - GeoDrive sets a new benchmark in autonomous driving by enhancing action controllability and spatial accuracy through explicit trajectory control and direct visual condition input, while also supporting applications like non-ego vehicle perspective generation and scene editing [41].