VLM

Search documents
70K?端到端VLA现在这么吃香!?
自动驾驶之心· 2025-07-21 11:18
Core Viewpoint - End-to-end (E2E) autonomous driving is currently the core algorithm for mass production in intelligent driving, with significant advancements in the VLA (Vision-Language Architecture) and VLM (Vision-Language Model) systems, leading to high demand for related positions in the industry [2][4]. Summary by Sections Section 1: Background Knowledge - The course aims to provide a comprehensive understanding of end-to-end autonomous driving, including its historical development and the transition from modular to end-to-end approaches [21]. - Key technical stacks such as VLA, diffusion models, and reinforcement learning are essential for understanding the current landscape of autonomous driving technology [22]. Section 2: Job Market Insights - Positions related to VLA/VLM algorithms offer lucrative salaries, with 3-5 years of experience earning between 40K to 70K monthly, and top talents in the field can earn up to 1 million annually [10]. - The demand for VLA-related roles is increasing, indicating a shift in the industry towards advanced model architectures [9]. Section 3: Course Structure - The course is structured into five chapters, covering topics from basic concepts of end-to-end algorithms to advanced applications in VLA and reinforcement learning [19][30]. - Practical components are included to bridge the gap between theory and application, ensuring participants can implement learned concepts in real-world scenarios [18]. Section 4: Technical Innovations - Various approaches within end-to-end frameworks are explored, including two-stage and one-stage methods, with notable models like PLUTO and UniAD leading the way [4][23]. - The introduction of diffusion models has revolutionized trajectory prediction, allowing for better adaptability in uncertain driving environments [24]. Section 5: Learning Outcomes - Participants are expected to achieve a level of proficiency equivalent to one year of experience as an end-to-end autonomous driving algorithm engineer, mastering key technologies and frameworks [32]. - The course emphasizes the importance of understanding BEV perception, multimodal models, and reinforcement learning to stay competitive in the evolving job market [32].
2025中国高阶智能辅助驾驶最新技术洞察:算力跃迁、数据闭环、VLA与世界模型
EqualOcean· 2025-06-05 05:42
Investment Rating - The report does not explicitly state an investment rating for the industry Core Insights - The report highlights the evolution of advanced driver assistance systems (ADAS) in China, focusing on the expansion of operational design domains (ODD), technological equity, safety concerns, and supportive policies [4][21][23] - It emphasizes the need for algorithm, data, and computing power upgrades to address safety shortcomings in high-level ADAS technologies [23][66] - The report discusses the transition from modular to end-to-end architectures in vehicle algorithms, aiming for human-like driving capabilities [66][68] Summary by Sections 1. Market Background - The expansion of high-level ADAS ODD is noted, with a focus on technological inclusivity and addressing accident anxiety through safety redundancies [4][21] - Policy support is highlighted as crucial for rational promotion of ADAS technologies [4][21] 2. Technology Insights - The report decodes the underlying logic of data, algorithms, and computing power in high-level ADAS [4][28] - It discusses the computing power landscape, noting the shift towards higher TOPS (trillions of operations per second) capabilities in vehicle and cloud computing [42][44] - Data challenges, including collection and positioning technologies, are identified as critical areas for development [4][28] 3. Competitive Analysis - The competitive landscape is analyzed, detailing the tiered structure of companies and their development strategies [29][30] - The report outlines various collaboration models among automotive manufacturers and technology providers, emphasizing the balance between self-research and external sourcing [83] 4. Trend Insights - The report notes the commercialization progress of passenger vehicle L3 systems, indicating a growing market for advanced ADAS [31][32] - It highlights the importance of continuous upgrades and iterations in ADAS functionalities to meet evolving consumer expectations and safety standards [82][83]
AI 如何成为理想一号工程
晚点LatePost· 2025-05-23 07:41
Core Viewpoint - The article discusses Li Auto's strategic focus on artificial intelligence (AI) and its evolution from a vehicle-centric AI assistant to a multi-platform intelligent application, emphasizing the importance of AI in future competitiveness [4][5][6]. Group 1: Strategic Meetings and AI Prioritization - Li Auto holds biannual closed-door strategy meetings to discuss future directions, with significant participation from top executives and industry leaders [3]. - Following a strategic meeting, Li Auto adjusted its AI-related business priorities, emphasizing the strategic importance of intelligent driving over other AI applications [4][5]. - The company aims to become a global leader in AI by 2030, with a clear focus on enhancing its AI capabilities and applications [5][6]. Group 2: Development of AI Capabilities - Li Auto has transitioned its AI assistant, "Li Xiang," from a vehicle-only application to a multi-platform tool, including mobile and web applications [7]. - The company has invested in self-developed algorithms, achieving a full switch to in-house technology for its AI functionalities by March 2023 [7][8]. - The introduction of the multi-modal cognitive model, Mind GPT 1.0, marks a significant advancement in Li Auto's AI capabilities [7]. Group 3: Intelligent Driving and Technological Advancements - Li Auto's intelligent driving system, AD Max, was launched to address product shortcomings and enhance competitive positioning in the market [10][11]. - The company has initiated a large-scale recruitment drive for its intelligent driving team, reflecting its commitment to advancing this technology [10]. - The shift towards an "end-to-end" model for intelligent driving aims to streamline processes and improve system performance through better data utilization [10][11]. Group 4: Organizational Changes and AI Integration - Li Auto established an AI Technical Committee to integrate AI capabilities across various business lines, enhancing collaboration and execution [15][16]. - The committee includes leaders from key departments, ensuring that AI is a core focus in strategic decision-making [16][17]. - The company aims to develop a foundational model that serves as a core capability for all AI projects, positioning itself as a leader in the automotive AI landscape [17].
TransDiffuser: 理想VLA diffusion出轨迹的架构
理想TOP2· 2025-05-18 13:08
Core Viewpoint - The article discusses the advancements in the field of autonomous driving, particularly focusing on the Diffusion model and its application in generating driving trajectories, highlighting the differences between VLM and VLA systems [1][4]. Group 1: Diffusion Model Explanation - Diffusion is a generative model that learns data distribution through a process of adding noise (Forward Process) and removing noise (Reverse Process), akin to a reverse puzzle [4]. - The model's denoising process involves training a neural network to predict and remove noise, ultimately generating target data [4]. - Diffusion not only generates the vehicle's trajectory but also predicts the trajectories of other vehicles and pedestrians, enhancing decision-making in complex traffic environments [5]. Group 2: VLM and VLA Systems - VLM consists of two systems: System 1 mimics learning to output trajectories without semantic understanding, while System 2 has semantic understanding but only provides suggestions [2]. - VLA is a single system with both fast and slow thinking capabilities, inherently possessing semantic reasoning [2]. - The output of VLA is action tokens that encode the vehicle's driving behavior and surrounding environment, which are then decoded into driving trajectories using the Diffusion model [4][5]. Group 3: TransDiffuser Architecture - TransDiffuser is an end-to-end trajectory generation model that integrates multi-modal perception information to produce high-quality, diverse trajectories [6][7]. - The architecture includes a Scene Encoder for processing multi-modal data and a Denoising Decoder that utilizes the DDPM framework for trajectory generation [7][9]. - The model employs a multi-head cross-attention mechanism to fuse scene and motion features during the denoising process [9]. Group 4: Performance and Innovations - The model achieves a Predictive Driver Model Score (PDMS) of 94.85, outperforming existing methods [11]. - Key innovations include anchor-free trajectory generation and a multi-modal representation decorrelation optimization mechanism to enhance trajectory diversity and reduce redundancy [11][12]. Group 5: Limitations and Future Directions - The authors note challenges in fine-tuning the model, particularly the perception encoder [13]. - Future directions involve integrating reinforcement learning and referencing models like OpenVLA for further advancements [13].