Depth and motion discontinuities
Carlo Tomasi, Stanley T. Birchfield · 1999
Depth and motion discontinuities arise wherever a light ray incident on a camera sensor meets a discrete change in the depth or motion of the surfaces in the world. Because these discontinuities tend to coincide with occlusions and with the boundaries of objects, they provide useful information for a number of applications in computer vision, such as camera control, compression, and tracking. Moreover, because they have simple, precise definitions depending only upon the physics of the scene, they are unaffected by subjective considerations. The first part of this thesis presents an algorithm to detect depth discontinuities from a stereo pair of images by matching pixels in corresponding scanlines and then propagating information between those scanlines. It uses a new measure of pixel dissimilarity that is provably insensitive to image sampling. The algorithm is fast and is shown to produce good results on difficult images containing untextured, slanted surfaces. Then some work aimed at detecting motion discontinuities from a monocular image sequence is described, along with a discussion of why this is a harder problem. A preliminary algorithm is given which tracks sparse features throughout a sequence, groups them according to an affine motion model, and traces the boundaries between the groups. Results show that the technique is promising but needs future work. Then two recent maximum-flow-based stereo algorithms are compared, showing their ability to work well on textured, fronto-parallel surfaces as well as their inability to handle untextured, slanted surfaces. We extend one of the techniques to solve stereo or motion correspondence with textured, slanted surfaces. Finally, an algorithm that uses the knowledge of discontinuities to track a person's head is described. Modeling the head as an ellipse and concentrating on the intensity edges around the perimeter as well as the color of the interior, the system is able to automatically control a camera in real time to follow a person moving around an unmodified environment. Extensive experimentation shows the algorithm's robustness with respect to full 360-degree out-of-plane rotation, severe but brief occlusion, arbitrary camera movement, and a dynamic background.