PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation
Frederic Besse, Carsten Rother, Andrew W Fitzgibbon, Jan Kautz · 2012
PatchMatch is a simple, yet very powerful and successful method for optimizing continuous labelling problems.The algorithm has two main ingredients: the update of the solution space by sampling and the use of the spatial neighbourhood to propagate samples.We show how these ingredients are related to steps in a specific form of belief propagation in the continuous space, called Particle Belief Propagation (PBP).However, PBP has thus far been too slow to allow complex state spaces.We show that unifying the two approaches yields a new algorithm, PMBP, which is more accurate than PatchMatch and orders of magnitude faster than PBP.To illustrate the benefits of our PMBP method we have built a new stereo matching algorithm with unary terms which are borrowed from the recent PatchMatch Stereo work and novel realistic pairwise terms that provide smoothness.We have experimentally verified that our method is an improvement over state-of-the-art techniques at sub-pixel accuracy level. IntroductionThis paper draws a new connection between two existing algorithms for estimation of correspondence fields between images: Belief Propagation [15,19] and PatchMatch [1, 2].Correspondence fields arise in problems such as dense stereo reconstruction, optical flow estimation, and a variety of computational photography applications such as recoloring, deblurring, high dynamic range imaging, and inpainting.By analysing the connection between the methods, we obtain a new algorithm which has performance superior to both its antecedents, and in the case of stereo matching, represents the current state of the art on the Middlebury benchmark at sub-pixel accuracy.The first contribution of our work is a detailed description of PatchMatch and belief propagation in terms that allow the connection between the two to be clearly described.This analysis is largely self-contained, and comprises the first major section of the paper.Our second contribution is in the use of this analysis to define a new algorithm: PatchMatch Belief Propagation (PMBP) which, despite its relative simplicity, is more accurate than PatchMatch and orders of magnitude faster than PBP.