华科&小米SparseOccVLA：统一的4D场景理解预测和规划，nuScenes新SOTA......

Core Insights - The article discusses the development of SparseOccVLA, a new Vision-Language-Action model that effectively bridges the gap between Vision Language Models (VLMs) and Semantic Occupancy, addressing challenges in autonomous driving scenarios [2][3][32] Group 1: Model Development - SparseOccVLA utilizes a lightweight Sparse Occupancy Encoder to generate compact yet information-rich sparse occupancy queries, serving as the sole bridge between visual and language inputs [3][14] - The model integrates a language model-guided Anchor-Diffusion planner, which features decoupled anchor scoring and denoising processes, significantly enhancing planning performance and stability [3][20] Group 2: Performance Metrics - SparseOccVLA demonstrates superior performance in various benchmarks, achieving a 7% relative improvement in the CIDEr metric on the OmniDrive-nuScenes dataset compared to the current best methods [3][23] - In the Occ3D-nuScenes dataset, SparseOccVLA also surpasses state-of-the-art performance in future occupancy prediction [24] Group 3: Technical Challenges - Traditional VLMs face issues such as token explosion and limited spatiotemporal reasoning capabilities, while Semantic Occupancy models struggle with dense representations that are difficult to integrate with VLMs [4][9] - The article highlights the limitations of existing methods in effectively combining VLMs and occupancy models, which have developed independently in the autonomous driving field [4][11] Group 4: Experimental Results - The experimental results indicate that SparseOccVLA requires significantly fewer tokens (as low as 300) to achieve competitive performance compared to methods that require over 2500 tokens, ensuring efficient inference [23] - The model's ability to recognize both tangible objects and non-geometric elements, such as traffic lights and lane markings, is attributed to its end-to-end design that retains visual signals from the original images [31]