Relaxed Relational Knowledge Distillation for Inter–Intra Class Learning in Human Activity Recognition
Li Wang, Lei Zhang, Jianglai Yu, Dongzhou Cheng, Minghui Yao, Hao Wu, Aiguo Song · IEEE Sensors Journal · 2024
Human activity recognition (HAR) has increasingly attracted interest within the ubiquitous computing community. It primarily focuses on improving model effectiveness and precision for deployment on devices with limited processing capabilities, essential for real-time applications. However, current deep learning (DL) approaches in HAR encounter three main challenges: high computational demand, overfitting due to complex model architectures, and the difficulty of transferring deep model insights to simpler, deployable models. To fully address these issues, this article introduces a pioneering relaxed relational knowledge distillation (RRKD) approach named RRKD, that seeks to capture relational dynamics in sensor data using a robust, well-optimized, and computationally modest fully convolutional network. Our extensive experiments confirm that this method surpasses existing state-of-the-art algorithms in achieving superior accuracy and operational efficiency without requiring extensive hardware resources, thereby bridging a critical gap between knowledge distillation (KD) and efficient activity recognition. The visualization analysis demonstrates that our technique effectively identifies and interprets underlying patterns and connections in time series sensor data, indicating significant potential in capturing the nuances of sensor data. The implementation has been successfully evaluated on an embedded platform, and the code will be made available athttps://github.com/legalstark/RRKD.