Hand Gesture Recognition Using Glove Mounted Sensor Data
Szabolcs Hanzel, Tamás Kari, Dávid Varga, Katalin Péter, Ferenc Füstos, Csaba Sulyok · 2024
This paper presents a method to recognise fine motor hand movements and visualise them in virtual space by using a smart glove. Sensors mounted on the glove provide data to determine the position of the hands. The presented system is capable of real-time motion recognition by processing continuously incoming sensor data, making it suitable for solving complex problems such as interaction with the VR/ AR world, sign language translation or even as a training tool for physiotherapists. A microcontroller-based glove is built for the project, which monitors head and hand position as well as finger joint curvature, drawing inspiration from open source projects such as OpenGloves and LucidVR. Sensor data from the Oculus Rift VR glasses and the glove is extracted and displayed using the SteamVR extension in the Unity video game engine. The data is streamed to the gesture recognition server, which classifies the motion into one of the predefined hand motion classes. The paper demonstrates the practical utility of the smart glove through a simulated traffic management system. The user can control traffic using the glove by predefined hand movements. The virtual space is based on the U nity3D game engine and Unity VR technology.