背景与动机 本文只做学术分享，如有侵权，联系删文 >> 点击进入→ 具身智能之心 技术交流群 更多干货，欢迎加入国内首个具身智能全栈学习社区 ： 具身智能之心知识星球 (戳我) ， 这里包含所有你想要 的。 当前视觉语言模型（VLMs）在空间推理任务（如物体位置/大小比较、多视角关系理解）中存在显著不 足。现有数据集存在三大局限： 点击下方 卡片 ，关注" 具身智能 之心 "公众号 核心贡献 作者丨 Nianchen Deng等 1. InternSpatial数据集 编辑丨具身智能之心 规模与结构 ： 指令多样性 ：支持19种指令格式（Table 1对比） 1. 场景单一性 ：数据源集中于室内/室外场景（如SpatialVLM、OSD），缺乏驾驶、具身导航等多样化环 境； 2. 指令格式受限 ：仅支持自然语言或区域掩码（如SpatialQA仅用文本，OSD依赖掩码），难以覆盖真实 应用中的多样化查询形式； 3. 多视角监督缺失 ：现有数据聚焦单图推理（占比超90%），缺乏跨视角时空关系建模能力。 视觉格式 ：原始图/带边界框图/掩码图/编号物体图（Figure 2示例） 1200万QA对（950万单视图 + ...

Core Insights - The article discusses the development of the ViLaSR-7B model, which enhances spatial reasoning capabilities in large vision-language models (LVLMs) through a novel "Drawing to Reason in Space" paradigm, achieving significant improvements in various spatial reasoning tasks [1][17][33]. Group 1: Model Performance - ViLaSR-7B achieved an average improvement of 18.4% across five major spatial reasoning benchmarks, including maze navigation and video spatial reasoning [3][25]. - The model reached a 45.4% accuracy on the VSI-Bench, outperforming the Qwen2.5-VL-7B by 12.7% [26]. Group 2: Training Framework - The model employs a three-stage training framework: 1. Cold-start training establishes basic visual operation capabilities [22]. 2. Reflective rejection sampling enhances self-correction and reflection abilities [23]. 3. Reinforcement learning optimizes overall reasoning capabilities and drawing operation efficiency [24]. Group 3: Reasoning Paradigms - The article highlights a shift from the traditional "visual-to-text" reasoning paradigm to the "Thinking with Images" paradigm, which allows models to actively manipulate images during reasoning [10][15]. - This new paradigm addresses limitations in the traditional approach, such as loss of critical details and temporal information during the visual encoding process [11][16]. Group 4: Human-like Reasoning Strategies - ViLaSR-7B demonstrates human-like spatial reasoning strategies, such as reference-based measurement reasoning and systematic cross-frame object tracking [30][32]. - The model's ability to identify and utilize reference objects for accurate measurements reflects a mature reasoning process similar to human problem-solving [31].

Core Viewpoint - The article discusses the transition of Visual Language Models (VLM) from "perception" to "cognition," highlighting the introduction of "Pixel-Space Reasoning" which allows models to interact with visual information directly at the pixel level, enhancing their understanding and reasoning capabilities [1][2][3]. Group 1: Key Developments in VLM - The current mainstream VLMs are limited by their reliance on text tokens, which can lead to loss of critical information in high-resolution images and dynamic video scenes [2][4]. - "Pixel-Space Reasoning" enables models to perform visual operations directly, allowing for a more human-like interaction with visual data [3][6]. - This new reasoning paradigm shifts the focus from text-mediated understanding to native visual operations, enhancing the model's ability to capture spatial relationships and dynamic details [6][7]. Group 2: Overcoming Learning Challenges - The research team identified a "cognitive inertia" challenge where the model's established text reasoning capabilities hinder the development of new pixel operation skills, creating a "learning trap" [8][9]. - To address this, a reinforcement learning framework was designed that combines intrinsic curiosity incentives with extrinsic correctness rewards, encouraging the model to explore visual operations [9][12]. - The framework includes constraints to ensure a minimum rate of pixel-space reasoning and to balance exploration with computational efficiency [10][11]. Group 3: Performance Validation - The Pixel-Reasoner, based on the Qwen2.5-VL-7B model, achieved impressive results across four visual reasoning benchmarks, outperforming models like GPT-4o and Gemini-2.5-Pro [13][19]. - Specifically, it achieved an accuracy of 84.3% on the V* Bench, significantly higher than its competitors [13]. - The model demonstrated a 73.8% accuracy on TallyQA-Complex, showcasing its ability to differentiate between similar objects in images [19][20]. Group 4: Future Implications - The research indicates that pixel-space reasoning is not a replacement for text reasoning but rather a complementary pathway for VLMs, enabling a dual-track understanding of the world [21]. - As multi-modal reasoning capabilities evolve, the industry is moving towards a future where machines can "see more clearly and think more deeply" [21].

ReasonMap团队 投稿 量子位 | 公众号 QbitAI 近年来，大语言模型（LLMs）以及多模态大模型（MLLMs）在多种场景理解和复杂推理任务中取得突破性进展。 然而，一个关键问题仍然值得追问： 多模态大模型（MLLMs），真的能"看懂图"了吗？ 特别是在面对结构复杂、细节密集的图像时，它们是否具备细粒度视觉理解与空间推理能力，比如挑战一下高清 地铁图 这种。 为此，来自西湖大学、新加坡国立大学、浙江大学、华中科技大学的团队提出了一个全新的评测基准 ReasonMap 。 看得出来北京、杭州的地铁图难倒了一大片模型。 这是首个聚焦于 高分辨率交通图（主要为地铁图）的多模态推理评测基准，专为评估大模型在理解图像中细粒度的结构化空间信息 方面的 能力而设计。 结果发现，当前主流开源的多模态模型在ReasonMap上面临明显性能瓶颈，尤其在 跨线路路径规划 上常出现视觉混淆或站点遗漏。 而经强化学习后训练的闭源推理模型（如 GPT-o3）在多个维度上 显著优于 现有开源模型，但与人类水平相比仍存在明显差距。 在面对不同国家地区的地铁图中，四个代表性 MLLM（Qwen2.5-VL-72B-I（蓝色）、 I ...

ZJU REAL Lab 投稿 量子位 | 公众号 QbitAI 杯子在我的左边还是右边？ 这个对人类来说非常简单的问题，连GPT-4o这样级别的视觉语言大模型 （VLMs） 也可能答错。 ViewSpatial-Bench评估集中 包含5700个问答对，涵盖相机视角与人类视角两种框架下的五种空间定位识别任务 。 究其根本，还是 当前的视觉语言大模型在大规模图文数据中学习到的空间信息往往是片段化的，仅限于静态视角的理解，缺乏多维度、多视 角的空间推理能力 。 因此，当面对需要多视角空间推理的任务时，这些模型们就频频卡壳。 但是，具备稳健的空间推理能力与视角理解能力的AI系统，才能真正成为与人类协作的智能体。 为此，来自浙江大学、电子科技大学和香港中文大学的研究团队提出了 首个系统评估VLM多视角多任务下的空间定位能力的基准体系 —— ViewSpatial-Bench，涵盖五种不同的任务类型，从相机和人类视角出发，全面评估模型的空间推理能力。 同时还并配备了能够生成精确方向标签的自动化3D标注流水线。通过高效的3D方向标注生成流程，实现了超过5700个问答对，覆盖丰富的 3D场景。 通过在多视角空间数据集上的 ...