Workflow
GazeGrasp系统
icon
Search documents
戴上眼镜,就能控物!俄罗斯团队让机械臂“秒懂”你的眼神
机器人大讲堂· 2025-10-23 14:37
Core Viewpoint - The GazeGrasp system developed by Skoltech allows users to control a robotic arm using only their gaze, providing significant assistance to individuals with severe motor disabilities. The introduction of the "magnetic effect" feature has improved operational efficiency by 31% [2][14]. Group 1: Technology Integration - The GazeGrasp system integrates multiple advanced technologies, including eye-tracking hardware and deep learning algorithms, to interpret user gaze and control the robotic arm [4][6]. - A specialized pair of glasses equipped with an ESP32 CAM camera module captures eye movements in real-time and transmits data wirelessly [4]. - The system utilizes Google's MediaPipe framework for facial feature tracking and YOLOv8 for object detection, ensuring accurate identification of common items [6][7]. Group 2: Magnetic Effect and Efficiency - The "magnetic effect" feature allows the cursor to snap to the center of an object when the user's gaze enters its boundary box, significantly reducing the difficulty of precise targeting [7]. - Experimental data shows that the average time to lock onto a target object decreased from 6.77 seconds to 4.65 seconds with the magnetic effect, demonstrating a substantial improvement in user efficiency [2][15]. Group 3: Coordinate Transformation - The system employs a precise coordinate transformation algorithm to convert screen pixel coordinates into real-world coordinates for the robotic arm [8][9]. - A one-time calibration process is required, where users gaze at 35 preset points to establish a mapping between eye movements and screen coordinates [8]. Group 4: User Interaction and Applications - The entire operation process involves the user gazing at the target object for three seconds, prompting the robotic arm to move and grasp the object, followed by gazing at a blank space to place it [11]. - GazeGrasp represents a significant innovation in human-machine interaction, allowing individuals with severe disabilities to regain independence in daily tasks [14]. - The technology has potential applications beyond healthcare, including industrial automation and remote operations in hazardous environments [14].