Sensor Fusion for Physical Exercises Classification
Yanhua Zhao, Sebastian Dill, Arjang Ahmadi, Martin Grimmer, Dennis Haufe, Luis Herrmann, Maziar Ahmad Sharbafi, Christoph Hoog Antink · 2025
With the rapid development of computer vision, video-based classification of physical activity has become a powerful tool in human-computer interaction, medical care, sports and other fields. This paper investigates how the integration of multiple sensors can improve the performance of movement classification. Using multiple sensors, user movements are captured in real-time and a machine learning model is used to identify the correctness of specific movements, e.g. squatting. The ability of the model to accurately recognise the movement when the movement is not completed is investigated. The distribution of feature importance in the time domain is analysed, as well as the importance of each joint. This research shows that using two video cameras leads to more stable results and vastly improves the motion classification compared to a single camera. The achieved average accuracy is comparable to the one reached with MoCap data, while being more convenient to set up. The model requires only 30 frames of data to achieve highly accurate classification without waiting for the complete movement to be executed. According to the feature importance analysis, data between 14th and 23rd frames of the sample had a higher impact on the classification decision of the model. This indicates that the temporal dynamics are essential in the classification of movements. This research could contribute to enhancing user experience through interactive and personalised exercises.