Yoga Pose Classification Based on Transfer Learning of Vision Transformer
Mingyu Fang · 2023
Yoga is gaining popularity for fitness and medical purposes due to its benefits in many aspects. However, inappropriate performance of yoga poses may have some disadvantageous effects. To help reduce inappropriate performance, the computer vision method, which is helpful in recognizing different types of yoga poses to provide suggestions and guidance, can be employed. In this work, a dataset was created by combining two existing datasets. Considering the relatively small scale of the dataset, transfer learning was used to perform yoga pose classification task. The Vision Transformer model was selected, fine-tuned, and evaluated for classifying yoga poses, reaching 92.61% accuracy and 92.62% F1 score. Compared with several typical CNN-based models of different scales, including GoogLeNet, ResNet, Inception, DenseNet, ShuffleNet, MobileNet, EfficientNet, the Vision Transformer had the best performance in every metric used in this work, including accuracy, precision, recall and F1 score, making it possible to be applied to real yoga pose classification tasks. Notably, due to the largest number of the parameters of Vision Transformer, reducing the parameters of Vision Transformer may be one of the research directions in the future.