Pose Based Pedestrian Street Cross Action Recognition in Infrared Images
Raluca Didona Brehar, Cristian Vancea, Mircea Paul Muresan, Sergiu Nedevschi, Radu Gabriel Danescu · 2021
In the context of a traffic scenario captured during night with infrared cameras we focus on pedestrian street cross action and we study the influence of the pedestrian pose with respect to the road environment on the accuracy of the action recognition model. This paper presents a complete framework that performs pedestrian cross action recognition for infrared sequences captured mainly during night but also during day time. The main contribution of the paper resides in the study of the variation in pedestrian action recognition accuracy provided by the combination of pedestrian pose-based features with several road context features given by semantic segmentation networks. The main modules of the proposed framework consist in a YOLO based infrared pedestrian detector combined with a tracking algorithm that enhances the detections. A CNN based pose estimator is applied on detected pedestrians in order to extract the relevant keypoints of the pedestrian skeleton. Several semantic segmentation networks like U-Net, FCN and PSPNet have been adapted in order to perform the semantic segmentation of the road in infrared images. Pose features are combined with road context features provided by the semantic segmentation and input to a LSTM based cross action recognition network. The obtained results provide a 90% accuracy on CROSSIR dataset.