Enhancing Piano Practice Techniques: A Deep Learning System for Front Sitting Posture and Side Fingering Recognition
Yen-Chiu Chen, Cong-You Lin, Yi-Cheng Chiou, Yu-Chen Chen, Mao-Sen Chen, J. H. Lin · 2024
Prolonged maintenance of poor posture not only leads to physical fatigue and discomfort but may also result in severe bodily injuries, such as spinal curvature and muscle problems. To addresses posture-related issues, this paper proposes a Body Posture Image Recognition App System based on edge computing technology, focusing on piano playing posture as an example. Users can capture their piano playing posture in real-time through the smartphone with a camera, incorporating both frontal sitting posture detection and lateral hand detection. This comprehensive approach aids in the effective correction and maintenance of proper sitting and wrist angles, ultimately enhancing performance and technique. For frontal sitting posture detection, this study employs the Movenet algorithm from Tensorflow Lite, utilizing 863 carefully selected images divided into training and testing sets, achieving a mean of Average Precision (mAP) of 74% for posture recognition. Additionally, the system includes hand recognition functionality utilizing the Mediapipe Pose algorithm. The technology developed in this study is intended for application among piano learners, positively impacting piano students and individuals requiring prolonged adherence to specific postures.