Mobile real-time gesture detection application for sign language learning
Liliia Ivanska, Tetyana Korotyeyeva · 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT) · 2022
An educational gesture detection system for real-time sign language learning was developed. The applied 3-step approach consists of continuous image gathering, hand detection using the MediaPipe pipeline from Single-Shot Detector BlazePalm method and Hand Landmark Model, pretrained on a combination of synthetic and real images, for hand key points defining, and simple gesture classification. Its primary purpose is to provide an opportunity for people with hearing or speech disorders to improve their sign language skills with the help of a real-time gesture detection educational system. The process of sign recognition is divided into several phases: gathering an image from the camera, detecting a palm, setting key points, and defining a letter.