Deep learning-inspired image quality enhancement
Ruxin Wang · UTS ePRESS (University of Technology Sydney) · 2017
Enhancing image quality is a classical image processing problem that has received plenty of attention over the past several decades.A high-quality image is always expected in various vision tasks, and degradations such as noise, low-resolution, and blur are required to be removed.While the conventional techniques for this task have achieved great progress, the recent top performer, deep models, can substantially and significantly boost performance compared with conventional ones.The advantages of deep learning which enables it to achieve such success are its high representational capacity and the strong nonlinearity of the models.In this thesis, we explore the development of advanced deep models for image quality enhancement by researching several fundamental issues with different motivations.In particular, we are first motivated by a pivotal property of the human perceptual system that similar visual cues can stimulate the same neuron to induce similar neurological signals.However, image degradations can result in the fact that similar local structures in images exhibiting dissimilar observations.While the conventional neural networks do not consider this important property, we develop the (stacked) non-local auto-encoder which exploits self-similar information in natural images for enhancing the stability of signal propagation in the network.It is expected that similar structures should induce similar network propagation.This is achieved by constraining the difference between the hidden representations of non-local similar image blocks during training.By applying the proposed model to image restoration, we then develop a "collaborative stabilisation" step to further rectify forward propagation. LIST OF FIGURES4.6 The SISR performances of the L20 models with different number of dilated convolutional layers. . . . . . . . . . . . . . . . . . . . .80 4.7 Examples of local entropy. . . . . . . . . . . . . . . . . . . . . . .82 4.8 Improved PSNR Δ L10D0 P s best v.s.average local entropy for each image in BSD100 and Urban100.In each case, the best model (L10D2 for ×2, L10D2 for ×3, and L10D3 for ×4) is used for evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83 4.9 SR examples of L10D2 for ×3.From top to bottom: original HR image, restored image, residual image, and local entropy image.The left two columns are the examples that achieve the highest Δ L10D0 P 3 L10D2 in BSD100 and Urban100, respectively.The right two columns achieve the lowest Δ L10D0 P 3 L10D2 in the two datasets.84 4.10 The SISR performances of the LmD0Po models for the investigation of model depth under the setting of an insufficient receptive field size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87 4.11 The SISR performances of the L6DnPo models for the investigation of model depth under the setting of a sufficient receptive field size.89 4.12 The SISR performances of the L10DnPo models for the investigation of model depth under the setting of a sufficient receptive field size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90 4.13 SR examples with an up-scaling factor of 3. PSNR/SSIM values are marked under each subfigure. . . . . . . . . . . . . . . . . . .96 4.14 SR examples with an up-scaling factor of 3 (cont.).PSNR/SSIM values are marked under each subfigure. . . . . . . . . . . . . . .97 4.15 SR examples with an up-scaling factor of 3 (cont.).PSNR/SSIM values are marked under each subfigure. . . . . . . . . . . . . . .98 5.1 Illustration of the image residual.(a) The original sharp image x.(b) The blur kernel k.(c) The blur image y.(d) The residual r b .(e) The pre-deconvolved image x.(f) The residual r. . . . . . . .103 5.2 The network architecture.The model takes a pre-deconvolved image as input and estimates the residual image.c = 3 for colour images and c = 1 for grey images. . . . .