A Novel Magnetometer Array-based wearable system for ASL gesture recognition
Prashanth Jonna, Madhav Rao · 2023
Hand gesture classification is of high importance in any sign language recognition (SLR) system, which is expected to assist individuals suffering from hearing and speech impairment. American sign language (ASL) comprises of static and dynamic gestures representing many alphabets, phrases, and words. ASL recognition system allows us to digitize communication and use it effectively within or outside the hearing-deprived community. Developing an ASL recognition system has been a challenge since some of the involved hand gestures closely resemble each other, and thereby it demands high discriminability features to classify these gestures. SLR through surface-based electromyography (sEMG) signals is computationally intensive to process and using inertial measurement units (IMUs) or flex sensors for SLR occupies too much space on the patient's hand. Video-based recognition systems place restrictions on the users by requiring them to make gestures or motions within the camera's field of view. A novel approach with a precision preserved static gesture classification system is proposed to fulfill the much-needed gap. The paper proposes an array of magnetometers-enabled static hand gesture classification system that offers an average accuracy of 98.60% for classifying alphabets and 94.07% for digits using the KNN classification model. The magnetometer array-based wearable system is devised to minimize the electronics coverage around the hand, and yet establish robust classification results that are useful for ASL recognition. The paper discusses the design of the proposed SLR system and also looks into optimizations that can be made to reduce the cost of the system.Clinical relevance - The proposed novel magnetometer array-based wearable system is cost-effective and works well across different hand sizes. It occupies a negligible amount of space on the user's hand and thus does not interfere with the user's everyday tasks. It is reliable, robust, and error-free for easy adoption towards building ASL recognition system.