Physiotherapy Exercise Classification by Exploring Knowledge Distillation with Attention Modules
Manik Hossain, Sumaiya Binte Siraj, Md. Mominul Haque, Gihun Song, Byungyong Ryu, Md. Tauhid Bin Iqbal · 2023
Automated AI-based physiotherapy assistive systems are gaining popularity owing to its practical necessity, particularly to the elderly people. The application of such an assistive system is highly reliant on the embedded AI model that needs to correctly classify exercise videos while being lightweight in order to be used for real-time purpose. In this paper, we take this issue into account, and present a deep learning framework for physiotherapy video classification task that preserves the competitively high accuracy while being lightweight as well. For this, we explore knowledge distillation techniques for model compression purpose. We use both the online and self-distillation techniques, and use Convolutional Long Short-Term Memory (ConvLSTM) & Long-Term Recurrent Convolutional Network (LRCN) as the networks. Moreover, we utilize different attention modules and explore different architectural combinations to further improve the classification performance. We conduct detailed experimentation by exploring different combination of distillation techniques and attention modules along with tweaking their hyperparameters as well. The best results found from our experiments are reported, and those combinations are recommended for further use while being utilized in relevant AI assistive systems.