Mutual Foreground Segmentation with Multispectral Stereo Pairs
Robert Bergevin, Pierre-Luc St-Charles, Guillaume-Alexandre Bilodeau · 2017
The foreground-background segmentation of video sequences is a low-level process commonly used in machine vision, and highly valued in video content analysis and smart surveillance applications. Its efficacy directly relies on the contrast between objects observed by the sensor. In this work, we study how the combination of sensors operating in the long-wavelength infrared (LWIR) and visible spectra can improve the performance of foreground-background segmentation methods. As opposed to a classic visible spectrum stereo pair, this multispectral pair is more adequate for foreground object segmentation since it reduces the odds of observing low-contrast regions simultaneously in both images. We show that by alternately minimizing a stereo disparity energy and a binary segmentation energy with dynamic priors, we can drastically improve the results of a traditional video segmentation approach applied to each sensor individually. Our implementation is freely available online for anyone wishing to recreate our results.