Depth estimation for stereo image pairs
Dion Boesten · TU/e Research Portal · 2009
Philips has developed a multi-view autostereoscopic 3D television which is capable of displaying video in real 3D without the use of external glasses or any other devices. Creating 3D content for these screens is difficult because they require not only the video itself but also the depth map as input. Depth maps are computed from a multi-view recording of the video sequence. To extract the depth map we have to solve the so called stereo correspondence problem. We developed a model, which with the help of some additional human input, can solve it reasonably well. We started by casting the original problem into the form of an energy minimization problem. The energy of a depth map consists of a weighted combination of two terms, a data term which indicates how well the depth maps corresponds with the input images, and an interaction term which is a measure of smoothness. The problem is then converted into the maximization of the probability of a Markov Random Field (MRF), a much used concept in the field of Computer Vision. An iterative procedure to obtain an approximate solution is derived from a Variational Mean Field approximation of the MRF. Several extensions are made to the resulting model in order to make it more suitable for the difficulties that arise in practical situations. These include a multi-scale approach, a gradient based automatic boundary detection algorithm, the possibility to incorporate (manually selected) occluded and boundary pixels in the computation of the depth map and the option of using input from three camera views instead of just two. Our implementation of the model allows for several ways of giving manual input in order to obtain improved depth maps, of which the most important is the manual correction of the automatically computed boundaries.