Pose-Guided Home Workout System via Graph-Based Multi-label Time Series Classification
Ming Chen, Suwang Xing, Yuqing Zhu, Xuxiang Ma, Zhenrong Luo · 2024
The COVID-19 pandemic has underscored the importance of home-based exercise. However, exercising without professional guidance often leads to incorrect postures, suboptimal results, and potential injuries. This paper introduces an intelligent workout system that provides real-time pose guidance and feedback by formulating pose guidance as a multi-label time series classification problem and leveraging graph-based techniques. Our system employs OpenPose to extract 2D skeletal data from workout videos, which is then transformed into 3D representations using GLA-GCN for a more comprehensive understanding of human poses. We manually annotate the 3D skeletal sequences with multiple labels indicating potential pose problems and corresponding adjustment suggestions. We then adapt the ST-GCN model for multi-label classification by introducing improvements to the model architecture and training process. The enhanced ST-GCN model effectively predicts necessary pose adjustments for each input video segment. Our system offers several advantages over previous approaches: it provides targeted and comprehensive feedback for a wide range of exercises without requiring exercise-specific algorithms; it is highly scalable, as it can be easily extended to cover more exercises and pose problems by augmenting the annotation dataset. Experiments on a diverse set of home workout videos demonstrate the effectiveness of our system in providing accurate and helpful pose guidance.