A Low-Jitter Hand Tracking System for Improving Typing Efficiency in Virtual Reality Workspace
Tianshu Xu, Wen Gu, Koichi Ota, Shinobu Hasegawa · 2023
Virtual reality technology has the potential to revolutionize immersive experiences in various applications, including office settings. However, efficient text entry in VR remains a significant challenge. This study addresses this challenge by proposing a machine learning-based solution, the 2S-LSTM typing method, to enhance text entry performance in VR. The 2S-LSTM leverages the back of the hand image. It employs a two-stream Long Short-Term Memory (LSTM) network, combined with a Kalman Filter (KF), to improve hand position tracking accuracy and reduce jitter. The results from questionnaire-based evaluations and typing data analysis demonstrate the superiority of the 2S-LSTM solution over existing solutions like Oculus Quest 2 and Leap Motion in terms of typing efficiency, fatigue reduction, accurate hand position replication, and positive user experience. These findings contribute to the advancement of text entry in VR environments and pave the way for immersive work experiences in the office and beyond.