Recognition and rehabilitation of hand function actions in stroke patients based on pose estimation
Shuyu Zhu, Tianyi Sun, Fanglve Yuan, Xiaofeng Lu · 2025
This paper proposes a method for recognizing and assessing hand function actions based on human pose estimation, aiming at the problem of daily assessment of hand function rehabilitation status in stroke patients. First, a pose estimation model based on ResNet-50 and COCO-WholeBody-Hand is used to extract the key points of the hand in the video of stroke patients completing standard movements, and the PoseC3D model is used for training and prediction in order to realize the hand function actions recognition. Next, a geometric feature structure-based assessment method is proposed to further evaluate the rehabilitation degree of the correctly recognized hand function actions. Specific evaluation indexes were designed for each standard movement, and a reasonable threshold was found as a quantitative assessment criterion with PSO algorithm. Experiments were conducted on a self-constructed dataset of hand function actions from stroke patients. The dataset includes nine categories of standard movements. With an accuracy of 99.60% in training progress, the PoseC3D model significantly outperforms traditional methods. Meanwhile, the method achieves an accuracy of over 80% for all movements in our dataset. This proves the effectiveness and accuracy of the recognition and evaluation methods proposed in this paper.