Online Violent Behavior Recognition Based on Lightweight Temporal Shift Convolutional Neural Network
Lujia Zhang, Haifeng Le, Jianjun Fang, Yanxia Liu · 2023
As a challenging task in the intelligent security monitoring system, violent behavior recognition plays a vital role in ensuring public safety. However, due to the complex semantic information about violent behaviors, there is still a lack of fast and reliable detection methods. To improve the accuracy and inference speed of violent behavior recognition, a lightweight convolutional neural network recognition algorithm named M-TSM based on the combination of temporal shift and spatial attention is proposed to solve the problems of insufficient motion information and slow inference speed in surveillance video images. First, random and equally spaced frames are drawn from the surveillance video to obtain the sequence of original video frames and normalized, then the depthwise separable convolution residual module is used to extract image features; the time shift module is used to fuse multiple frames of images to obtain video features containing multi-level time series information; finally, the spatial attention module focuses on the local features of the human body, so that the model can better distinguish between regular and violent videos, and output prediction results. The improved model is translated and compiled and then transplanted to the embedded device Jetson NX for deployment, which can realize edge computing, and the inference speed can meet the real-time requirements. The experimental results show that the recognition accuracy of the improved M-TSM algorithm in the violent behavior dataset is 84.1%, which is 1.4% higher than the original TSM algorithm, and the inference time in edge computing equipment is reduced by 25%, it has better real-time detection performance.