Réseaux de neurones convolutifs profonds pour problèmes inverses en restauration d'images et de vidéos
Benjamin Naoto Chiche · theses.fr (ABES) · 2022
Image and video restoration regroups numerous tasks---such as denoising, deconvolution, and super-resolution, to give a few examples---that enable applications that are of high interest in diverse research and industrial areas (textit{e.g.}, health, military, creative, and gaming industries and research in astrophysics). All restoration problems are modeled in the mathematical framework of inverse problems, in which forward models specify degradations connecting the observed corrupted data to its original data. These problems are classically solved based on hand-crafted regularizations to mitigate their ill-posedness and iterative algorithms that minimize sums of data fidelity and regularization terms. Deep learning (DL) and Convolutional Neural Networks (CNNs) have recently significantly increased image and video restoration performance. These networks can notably learn the regularization from data, textit{i.e.}, pairs of degraded and original data. The forward model is used in order to generate these pairs in this DL framework. Even though the learned regularization generally enables better performance than a hand-crafted one and CNNs are faster than iterative algorithms (thus more suitable for practical applications), CNNs are used as black boxes and lack interpretability. Moreover, they also lack flexibility in using knowledge of the forward model, contrary to classical inverse problem-solving. In some situations where the forward model is simple and well characterized, classical methods can still perform better than DL-based ones. Some more recent approaches are textit{hybrid}, blending the advantages of both methods in a complementary way. Some of them enable to design, for instance, a single and interpretable CNN that can flexibly manage knowledge about degradations.This work investigates neural network architectures to solve image and video restoration problems. First, we explain the principles of classical, DL-based, and hybrid image and video restoration methods. Second, we focus on the Video-Super-Resolution (VSR) inverse problem: we review its traditional solving and state-of-the-art solving based on DL. As our first contribution, we propose a hybrid VSR network that mixes the advantages of classical solving with the representation power of CNNs. As our second contribution, we propose a recurrent VSR network adapted for super-resolving long videos in which some parts of the scene barely move (this kind of video can be encountered in applications such as video surveillance) and introduce a new test dataset of such videos. Indeed, we demonstrate that existing recurrent VSR networks present instabilities on such videos. Finally, we focus on the deconvolution of image time-series in radio-interferometry, to enable better detection of transient astronomical sources. They are sources that appear and disappear over time and are highly interesting for astrophysicists because they are associated with high-energy physical phenomena. As our third contribution, we propose two neural network architectures that can do spatial and temporal modeling to solve this deconvolution problem.