Finger Gesture Recognition by Mediapipe Algorithm and Advanced YOLOV7 Network for Deaf People
Chi Cuong Vu, Thanh Long Nguyen, Thanh Hai Nguyen, Ba Viet Ngo · International Journal of Biomedical Engineering and Technology · 2024
Sign language is a challenge to be able to recognise and understand correctly its meaning. Therefore, with the deaf community, understanding sign language for daily activities is essential. For the support of the deaf community in interfacing together, researchers have represented different methods to support them in using sign languages. This article proposes using simple landmark hands with different finger shapes to detect ten letters in the American sign language (ASL) writing system. Therefore, an inference - MediaPipe algorithm for extracting hand features with the advanced 'you only look once version 7' (YOLOv7) network was applied for sign language recognition. The article used 2,000 hand images of five persons of different ages and genders with the YOLOv7 network to produce the high recognition performance. In particular, the average mAP accuracy reached 0.995, accuracy reached 99.4%, and recall parameter reached 99.2% using the confusion matrix. This algorithm can be developed with more letters.