Human Activities Recognition Based on FA-FlowNet and Multi-channel Weighted Fusion
Huimin Qian, Ruixin Shang, Zhijian Liu, Shi Chen · 2022 41st Chinese Control Conference (CCC) · 2022
Two-stream convolutional neural network (CNN) is one of the hot spots in human activities recognition from videos. In this paper, a new two-stream CNN is proposed by improving the optical flow extraction neural network and presenting a novel fusion method. Specifically, the proposed two-stream CNN is composed of dynamic-data-stream subnetwork (DDS-SN), static-data-stream subnetwork (SDS-SN), and fusion network. The input of DDS-SN is optical flow image sequence generated by the proposed FA-FlowNet from the video segments. And the input of SDS-SN is RGB image sequence of the video segments. In both DDS-SN and SDS-SN, the long-term recurrent convolutional network with GoogleNet is adopted to create the predicted recognition results. After that, the proposed multi-channel weighted fusion method is applied to integrate the results of DDS-SN and SDS-SN. The performance improvements of two-stream CNN by introducing the developed technologies, including FA-FlowNet, multi-channel weighted fusion method, and GoogleNet, have been demonstrated by experimental results.