Extracting 3D Trajectories of Objects from 2D Videos using Particle Filter
Zeyd Boukhers, Kimiaki Shirahama, Frédéric Li, Marcin Grzegorzek · 2015
Depth estimation is a method to estimate the depth information in a 2D image/video, where the original 3D space is projected onto an image plane. This paper introduces a novel extension of depth estimation in the video domain, where we extract 3D trajectories which individually represent the transition of an object in the 3D space. Such 3D trajectories are useful for appropriately characterising spatio-temporal object relations for video event detection. While we extract 3D trajectories by combining depth estimation and object detection results, the major problem is the inconsistency between these results. For example, significantly different depths may be estimated for the region of the same object, and an object region that is appropriately shaped by estimated depths may be missed. To overcome this, we first initialise the 3D position of an object using the frame with the highest consistency between the depth estimation and object detection results. Then, we track the object in the 3D space using particle filter, where a 3D position of the object is modelled as a hidden state to generate its 2D visual appearance. Experimental results demonstrate the effectiveness of our method.