ICCV 2025满分论文：一个模型实现空间理解与主动探索大统一

Core Insights - The article discusses the transition of artificial intelligence from the virtual internet space to the physical world, emphasizing the challenge of enabling agents to understand three-dimensional spaces and align natural language with real environments [3][40] - A new model proposed by a collaborative research team aims to unify spatial understanding and active exploration, allowing agents to build cognitive maps of their environments through dynamic exploration [3][40] Group 1: Model Overview - The proposed model integrates exploration and visual grounding in a closed-loop process, where understanding and exploration are interdependent and enhance each other [10][14] - The model consists of two main components: online spatial memory construction and spatial reasoning and decision-making, optimized under a unified training framework [16][22] Group 2: Exploration and Understanding - In the exploration phase, the agent accumulates spatial memory through continuous RGB-D perception, actively seeking potential target locations [12][21] - The reasoning phase involves reading from the spatial memory to identify relevant candidate areas based on task instructions, utilizing cross-attention mechanisms [22][23] Group 3: Data Collection and Training - The authors propose a hybrid strategy for data collection, combining real RGB-D scan data with virtual simulation environments to enhance the model's visual understanding and exploration capabilities [25] - The dataset constructed includes over 900,000 navigation trajectories and millions of language descriptions, covering various task types such as visual guidance and goal localization [25] Group 4: Experimental Results - The MTU3D model was evaluated on four key tasks, demonstrating significant improvements in success rates compared to existing methods, with a notable increase of over 20% in the GOAT-Bench benchmark [28][29] - In the A-EQA task, the model improved the performance of GPT-4V, increasing its success rate from 41.8% to 44.2%, indicating its potential to enhance multimodal large models [32][33] Group 5: Conclusion - The emergence of MTU3D represents a significant advancement in embodied navigation, combining understanding and exploration to enable AI to autonomously navigate and complete tasks in real-world environments [40]