Joint space-time motion-based video segmentation and occlusion detection using multiphase level sets
Mirko Ristivojevic, Janusz Konrad · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 2004
Spatial video segmentation is usually performed based on motion between two frames. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation can be interpreted as a volume "carved out" by a moving object in the image sequence domain, or the so-called "object tunnel". In this paper, we extend this concept to explicit modeling of occlusion events in space-time. In addition to the modeling of object evolution, we also explicitly model occluded and newly-exposed areas in the background by means of "occlusion volume", a new space-time concept. A voxel belongs to occlusion volume if its intensity is consistent with past intensities along its motion trajectory but inconsistent with future intensities (reversed for "exposed volume"). We propose a variational formulation of the problem that we solve using the multiphase level set method. We show encouraging experimental results for synthetic and natural image sequences.