Real-Time Video-Based Human Action Recognition on Embedded Platforms
Ruiqi Wang, Zichen Wang, Peiqi Gao, Mingzhen Li, Jaehwan Jeong, Yihang Xu, Yejin Lee, Carolyn Manville Baum, Lisa Tabor Connor, Chenyang Lu · ACM Transactions on Embedded Computing Systems · 2025
Advances in computer vision and deep learning have made video-based Human Action Recognition (HAR) increasingly feasible. However, running HAR on live video streams encounters significant delays on embedded platforms due to computational demands. This work addresses real-time HAR performance challenges through four key contributions: (1) an experimental study identifying standard Optical Flow (OF) extraction as the primary latency bottleneck in a state-of-the-art HAR pipeline, (2) an analysis of the latency-accuracy trade-off between traditional and deep learning-based OF methods, underscoring the need for an efficient motion feature extractor with minimal impact on accuracy, (3) the design of Integrated Motion Feature Extractor (IMFE) , a novel unified neural network architecture that substantially reduces motion feature extraction latency, and (4) the development of RT-HARE , a real-time HAR system optimized for embedded platforms. Experiments on three benchmark datasets of various characteristics using the Nvidia Jetson Xavier NX platform demonstrate that RT-HARE achieves real-time HAR with lower and more stable latency, reduced power consumption, and a smaller memory footprint while maintaining recognition accuracy comparable to more complex server-based HAR models.