Hand signal classification system for sign language communication in Virtual Reality
Octavian Dudas, Constantin Ioan Nandra, Cristinel Mihai Mocan, Dorian Gorgan · 2023
Sign language is an essential form of communication for people with hearing disabilities. However, effective communication traditionally required the use of trained translators. With recent advances in AI and virtual reality, technology seems to provide a viable means of facilitating communication with sign-language users. This paper describes a solution developed around the detection and classification of the user’s hand signals and shows its integration within a system meant to facilitate communication with hearing impaired individuals. The system relies on the capabilities of a virtual reality headset to capture the user’s hand signals and then employs a machine learning model in order to correctly classify them. Within this paper, we discuss some of the relevant technical aspects of the system as well as provide insight into its observed capabilities.