6. Image Inpainting
Society for Industrial and Applied Mathematics eBooks · 2005
Interpolation has been a significant topic in a number of areas including numerical analysis; computational PDEs; approximation theory; real, complex, and harmonic analysis; and signal processing. In image processing, image interpolation is such a fundamental problem that there have been numerous prior works in existence. In the engineering literature, for instance, one witnesses the following samples from a large pool: image interpolation [13, 181, 182], image replacement [156, 316], error concealment [161, 187], and image editing [111]. In mathematical image and vision analysis, image interpolation has also been studied systematically in the remarkable works of Nitzberg, Mumford, and Shiota for segmentation with depth and edge completion [234], Masnou and Morel for level-line completion [213, 214], and Caselles, Morel, and Sbert for axiomatic interpolation based on second order PDEs [50]. To our best knowledge, the work of Masnou and Morel [214], which won the best-student-paper award in ICIP '98, was the first work performing variational image interpolation as inspired by Nitzberg, Mumford, and Shiota's variational model for edge completion [234]. The word inpainting is an artistic synonym for image interpolation and has been circulated for quite a while among museum restoration artists [312]. It was first transplanted into digital image processing in the remarkable work by Bertalmio et al. [24], which has stimulated the recent wave of interest in numerous problems related to image interpolation, including the works by Chan and Shen and their collaborators. The current chapter presents several inpainting models based upon the Bayesian, variational, PDE, as well as wavelet approaches. Numerous applications of inpainting techniques in digital and information technologies are also discussed. The presentation of the current chapter attempts to organize the topics according to the logical mathematical structure espoused in Chapters 1, 2, and 3, i.e., from the general Baysian/variational principles to their associated Euler-Lagrange PDEs. We will also discuss other approaches that do not fit into this framework strictly but have been very important and successful, including the third order nonlinear inpainting PDE by Bertalmio et al. [24] and its Navier-Stokes interpretation [23], and the axiomatic approach of Caselles, Morel, and Sbert [50]. Inpainting of Markov random fields (as first heuristically employed in the well-known paper by Efros and Leung [110]) will also be briefly discussed in the end, and other more complex but powerful stochastic approaches to pattern-theoretic inpainting can be found, e.g., in the work of Guo, Zhu, and Wu [148].