Human Behavior Analysis Methods Based on Temporal Information Enhancement
Qing Ye, R.-B. Wang · 2023
Aiming at the difficulty in extracting temporal information from human behavior in videos, we propose a temporal information enhancement algorithm based on frame differencing. This algorithm computes the frame difference intensity through frame differencing and selects several frames with the highest frame difference intensity as key frames in the video. To enhance the temporal information for each frame, we employ frame differencing to capture the temporal differences between video frames. Then, channel descriptors are obtained through global average pooling, which increases the weights of frames related to motion and amplifies the subtle frame differences, facilitating the network to learn the temporal information of subtle movements. To effectively extract spatial and temporal features from videos, we utilize decomposed 3D convolutional kernels to extract spatiotemporal information. Experimental results on the Something-SomethingV1 dataset demonstrate that the proposed algorithm improves accuracy and temporal action detection compared to other video-based human behavior analysis algorithms.