Multi-Layer Machine Learning Knowledge Transfer Framework for Kinect Skeleton Joints Movements Classification
Mohammad Z. Masoud, Rashed Alsakarnah, Ahmad A. Manasrah, Yousef M. Jaradat · International Review of Automatic Control (IREACO) · 2024
The Internet of Things (IoT) has significantly transformed various aspects of people's lives, employing multiple sensors and devices for data collection to enhance control decisions and user experiences. Among these technologies, the Kinect sensor has emerged as a prominent tool in IoT applications, facilitating motion sensing and gesture recognition. However, existing systems often face challenges in recognizing new movements, requiring time-consuming retraining processes. To address this issue, a new multi-stage machine learning algorithm framework with knowledge transfer, leveraging Kinect skeleton node position data to characterize user movements is proposed. Our framework employs three parallel backpropagation Artificial Neural Network (ANN) models to identify hand, leg, and body movements, extracting five distinct movements from each. These movements are then inputted into a secondary neural network model to recognize complex movements. Importantly, our framework enables seamless incorporation of new movements without extensive retraining, utilizing the knowledge transfer mechanisms. The framework has been validated using a single Xbox Kinect V2 sensor, recording video data of 3 individuals performing various movements. Our results show that simple movement of different joints groups of the body can be categorized with accuracy up to 98%. Subsequently, complex movements are categorized with accuracy of approximately 92%. Finally, the knowledge transfer reduced the retraining complexity and an accuracy of 89.8% has been recorded for the new complex movements.