Core Viewpoint - The article discusses the introduction of a new technology called Reality Proxy, which enhances human-computer interaction by allowing users to seamlessly select and manipulate real-world objects through a mixed reality interface, overcoming limitations of traditional XR devices [10][13][14]. Group 1: Technology Overview - Reality Proxy is a digital representation of real-world objects that allows users to interact with them without being hindered by physical constraints such as distance or size [14][16]. - The interaction process involves three main steps: activating the proxy, generating the proxy, and interacting with the proxy [17][19][24]. - The system captures the semantic structure of the environment and creates proxies that maintain spatial relationships, allowing for intuitive manipulation [20][22]. Group 2: Interaction Features - Users can browse object previews, select multiple objects, filter objects by attributes, and utilize physical features for interaction [30][31][32][34]. - The technology supports semantic grouping and custom grouping, enabling users to organize and manipulate objects efficiently [36][40]. Group 3: Practical Applications - Reality Proxy can be applied in various scenarios, such as quickly locating specific books in an office or interacting with kitchen appliances [41][43]. - It facilitates efficient navigation and interaction in large buildings and allows for dynamic control of real-world objects like drones [45][47]. Group 4: User Feedback and Evaluation - Participants in the study found Reality Proxy to be practical and effective in addressing interaction challenges with distant or hard-to-reach objects [53]. - The system was praised for its speed and reduced physical fatigue, although some users noted a learning curve and the need for improved accuracy in proxy positioning [54][55].

时令 发自 凹非寺 量子位 | 公众号 QbitAI 只靠说话和AI眼镜互动，确实有点不方便。 现在新玩法来了！ 数字替身 帮你"隔空抓物"，混合现实即时选择现实世界的物体作为上下文。 书籍检索，轻松get～ 该技术被称为 Reality Proxy ——一种直接操控界面，让你可以瞬间选中现实世界中的物体。 研究者Xiaoan Liu甚至表示，这使我们离贾维斯又进了一步。 Reality Proxy打破物理条件限制 建筑导航？手拿把掐。 混合现实（XR）正在重塑人机交互的边界。通过头戴设备，它有望将物理世界与数字世界融合，让用户能同时操控现实与虚拟中的对象。 然而，传统的XR设备通常通过光线投射来选择物体，但由于目标在视野中尺寸较小、视线不稳定以及手部颤抖等原因，这一过程往往容易出 错。 多无人机协作？也是毫不逊色。 他们将交互目标从物体无缝转移到其代理上，选择代理等同于选择实际物体，便于用户摆脱距离或大小等限制，实现轻松选择物体。 如上所示，Reality Proxy流程包括三个主要步骤： 激活代理：捕获层级和语义场景结构 当用户捏合确认选择时，它会自动检测用户视线范围内的真实世界物体，并将其抽象为手部的交 ...