Recognizing fingerspelling in SIBI (sistem isyarat bahasa Indonesia) using OpenPose and elliptical fourier descriptor
Nanda Maulina Firdaus, Erdefi Rakun · 2019
Humans are social beings; they communicate with other humans to coexist. In general, humans use verbal methods to communicate. However, the limited of verbal communication in people with hearing loss causes them to use non-verbal communication through sign language. It is challenging for nondisabled to grasp sign language quickly because it takes a long time to learn and understand. Therefore, a system that can recognize SIBI is needed. This research focused on fingerspelling gesture in SIBI, concentrating on finger and hand movements. The accuracy results by using OpenPose as feature extraction achieved a 61.49% accuracy. To improve accuracy, by using OpenPose as hand tracking, image masking as pre-processing, and Elliptical Fourier Descriptor as feature extraction, this method achieved a 64.05% accuracy. We investigated whether feature extraction with EFD and OpenPose for hand tracking can improve accuracy; however, some labels still could not be detected correctly. To improve accuracy, image masking and delete similarity frames are used as preprocessing; OpenPose uses as hand tracking, and EFD uses as feature extraction. This method achieved 67.23% accuracy.