Human Kinematics Analysis by Markerless Vision Based on OpenSim
Chi Zhang, Yurong Li, Wangwei Ye, Gaohua Huang · 2024
Human Kinesiology analysis is essential for understanding biomechanical loads in rehabilitation, injury prevention, and diagnosis. However, traditional marker-based motion capture systems are suboptimal due to high equipment and time costs, as well as the need for specialized expertise. Although OpenSim can perform detailed kinematic analyses using musculoskeletal models, integrating these analyses seamlessly with sparse human key points derived from computer vision remains challenging. Additionally, the current triangulation methods based on direct linear transformation (DLT) have limitations in accuracy and generalization ability. While learnable triangulation methods can extract complex features from images and significantly improve the accuracy of identifying and locating human nodes, they still lack limb length constraints and require calibration for each inference. In order to solve the above problems, in this work, we apply a triangulation method combining graph convolutional network with joint context constraints and camera pose distribution to human kinematics analysis, which fully considers the bone length constraints and joint connection relations, and improves the reasoning ability of the model under different camera parameter datasets. In addition, we fine-tuned the backbone network using relevant data with foot markers to suit the needs of kinesiology analysis. Finally, the output key points of the triangulation network is sent into the deep learning network to obtain the corresponding anatomical markers, and the results in the visual field are better combined with the musculoskeletal model provided by OpenSim, and more accurate kinematic parameters are obtained.