Mask R-CNN based ski pose estimation
Wenqing Du, Tengfei Li, Chuanzheng Xu · 2023
Skiing as a winter sport is showing an increasing trend of popularity worldwide. With the success of the Beijing Winter Olympics, more and more people are participating in this project. Human pose estimation as an emerging research area in computer vision is applied to more and more industries. In this paper, human pose estimation is applied to skiing sports scene based on Mask R-CNN to achieve the detection of overall pose of snowboarding, which can be applied to skiing pose rating, skiing pose classification and so on. Unlike the general pose detection, the 17 keypoints of pose are added to the head and tail of the ski, which can show the overall pose of the skiing process in a more comprehensive way and provide convenience for the subsequent skiing action recognition and skiing action generation, and help the model to understand the data more. It can accomplish classification, bbox regression, instance segmentation, and keypoints prediction at the same time by adding different branches to provide more comprehensive information for application scenarios. The model automatically generates candidate regions by neural network, which is better than SS algorithm (Selective Search), and filters some invalid candidate regions in the Proposal layer, which reduces redundant computation and improves the speed. In this paper, the traditional Mask R-CNN is lightened and improved, and we try to use MobileNetV2 as the backbone network to replace the original ResNet-101 network, introduce the Inverted Residuals block and Linear Bottlenecks , reduce the number of model parameters, and improve the detection efficiency, so as to realize the lightened Mask R-CNN algorithm. Based on the existing work, this paper contemplates the use of PoseCnov3D for further prediction to achieve the recognition of eight basic skiing actions for skiing pose classification.