Recognition of Real-Time Angklung Kodály Hand Gesture using Mediapipe and Machine Learning Method
Muhammad Zulfikar Fauzi, Riyanarto Sarno · 2023
The application of machine learning in the arts and culture can help preserve these arts and culture and, in some cases even facilitate access to the arts to the public. In this study, a hand gesture recognition system was developed for the Kodaly method hand gesture to play angklung directly from a personal computer. Many methods of hand gesture recognition still need an additional hardware such as gloves and expensive sensors that may not be accessible to some people. This study proposes a computer vision-based approach utilizing the personal computer internal webcam without additional hardware and employs multi-layer perceptron classifier for real-time recognition and Mediapipe for feature extraction. The developed system achieves an average accuracy of 99% and runs with low latency between recognition and audio output. The effective range of hand gesture recognition is up to 1.5 meters in environment with a light level as low as 25 LUX.