Improving Finger Stroke Recognition Rate for Eyes-Free Mid-Air Typing in VR
Yatharth Singhal, Richard Huynh Noeske, Ayush Bhardwaj, Jin Ryong Kim · CHI Conference on Human Factors in Computing Systems · 2022
We examine mid-air typing data collected from touch typists to evaluate the features and classification models for recognizing finger stroke. A large number of finger movement traces have been collected using finger motion capture systems, labeled into individual finger strokes, and classified into several key features. We test finger kinematic features, including 3D position, velocity, acceleration, and temporal features, including previous fingers and keys. Based on this analysis, we assess the performance of various classifiers, including Naive Bayes, Random Forest, Support Vector Machines, and Deep Neural Networks, in terms of the accuracy for correctly classifying the keystroke. We finally incorporate a linguistic heuristic to explore the effectiveness of the character prediction model and improve the total accuracy.