Core Viewpoint - The article discusses the development of KnowVal, an advanced autonomous driving system that integrates perception and knowledge retrieval to enhance visual-language reasoning capabilities, aiming for higher levels of automated driving [4][21]. Group 1: System Overview - KnowVal is a novel autonomous driving system that combines perception modules with knowledge retrieval modules to achieve visual-language reasoning [4]. - The system constructs a comprehensive driving knowledge graph that includes traffic regulations, defensive driving principles, and ethical considerations, supported by an efficient retrieval mechanism based on large language models [4][15]. - KnowVal integrates a world model and a value model within its planner to ensure value-aligned decision-making [4][11]. Group 2: Technical Framework - The system employs an open 3D perception and knowledge retrieval framework, enhancing the traditional visual-language paradigm to facilitate basic visual-language reasoning [7][9]. - It utilizes specialized perception for autonomous driving and open-world 3D perception to extract both common and rare instance features, ensuring effective feature transfer throughout the system [9]. - The knowledge graph retrieval process involves natural language processing of perception data to access relevant knowledge entries, ranked by relevance [10][15]. Group 3: Value Model and Trajectory Planning - KnowVal's trajectory planning is based on a world prediction and value model, iteratively generating candidate trajectories and evaluating them against retrieved knowledge for value assessment [11][19]. - A large-scale driving value preference dataset was created to train the value model, consisting of 160,000 trajectory-knowledge pairs, which were annotated with value scores ranging from -1 to 1 [19]. Group 4: Experimental Results - The KnowVal framework was tested against baseline models GenAD, HENet++, and SimLingo, achieving the lowest collision rate on the nuScenes dataset and the highest driving score and success rate on the Bench2Drive benchmark [21]. - The results indicate that KnowVal outperforms existing end-to-end and visual-language-action models, demonstrating its effectiveness in real-world driving scenarios [21][22]. Group 5: Qualitative Analysis - The article highlights qualitative analysis examples to illustrate KnowVal's performance in adhering to legal and ethical driving behaviors, such as slowing down in wet conditions and obeying lane change regulations in tunnels [23][25].