A Lightweight Action Recognition Method for Deployable Embedded Devices for Human-Computer Interaction
Nanjie Hu, Ningyu Wang, Jie Lin, Qinghao Fu, Benying Tan · 2023
In recent years, numerous researchers have proposed various solutions to address the challenges in action recognition. However, most existing approaches suffer from high computational requirements and significant memory usage, making them impractical for real-time deployment on embedded devices. This paper introduces a lightweight action recognition method suitable for deploying on embedded devices for human-computer interaction, denoted as LARMDED-HCI (Lightweight Action Recognition for Mobile and Deployable Human-Computer Interaction). Our method employs MobileNetV3 as the backbone network, significantly reducing computational load and parameter count. We incorporate the Temporal Shift Module (TSM) to model the temporal aspect of video frames, enabling the model to capture temporal features effectively. Additionally, a 1D temporal convolutional layer is introduced to enhance feature extraction in the temporal dimension, improving the model’s ability to model temporal differences between adjacent frames. Experiments conducted on the Jester and Something-Something-V2 datasets demonstrate that our approach exhibits a noticeable competitive advantage in recognition accuracy compared to other methods. Furthermore, our method achieves satisfactory real-time performance when deployed on embedded devices, facilitating human-computer interaction on such platforms.