自动驾驶之心·2025-10-12 23:33Core Insights - The article discusses the emergence of two new players in the autonomous driving field: End-to-End (E2E) driving models and video generation models, highlighting their potential to simplify traditional systems and reduce testing costs [3][5] - A new framework called Drive&Gen is introduced, which aims to connect E2E driving models with generative world models for mutual evaluation and enhancement [6][8] Group 1: Background and Challenges - Traditional autonomous driving systems are complex and modular, while E2E models offer a streamlined approach by directly predicting driving decisions from raw sensor inputs [5] - The advancement of video generation models presents opportunities for creating "digital twin" environments for testing, but challenges remain in assessing the realism of generated videos and understanding the E2E model's decision-making process [5][6] Group 2: Drive&Gen Framework - Drive&Gen combines a controllable video generation model with an E2E driving planner, facilitating a collaborative evaluation process [8] - The framework utilizes a video diffusion model that can generate highly customized driving scenarios based on various conditions, such as weather and time of day [11] Group 3: Evaluation Metrics - A new evaluation metric called Behavioral Permutation Test (BPT) is proposed to assess the realism of generated videos, focusing on the driving decisions made by the E2E model [13] - BPT outperforms traditional metrics like Fréchet Video Distance (FVD) in capturing key differences that affect driving decisions, demonstrating its effectiveness in evaluating video quality [14][16] Group 4: Experimental Validation - Experiments show that the generated videos can lead to similar trajectory predictions as real videos, with a BPT failure rejection rate of 69.62%, indicating that the planner struggles to distinguish between real and generated videos [18] - The integration of synthetic data with real data significantly improves the E2E planner's performance, reducing the average displacement error (ADE@5s) from 0.7548 to 0.7333 [21] Group 5: Impact on Autonomous Driving - The framework allows for the creation of "out-of-distribution" scenarios, such as rainy and nighttime conditions, which are typically underrepresented in real-world data [21][23] - The results indicate that high-quality, controllable synthetic data can effectively supplement real-world data, enhancing the operational design domain of autonomous driving models [26]