Joint Depth and Density Guided Single Image De-Raining
Lei Cai, Yuli Fu, Tao Zhu, Youjun Xiang, Ying Zhang, Huanqiang Zeng · IEEE Transactions on Circuits and Systems for Video Technology · 2021
Single image de-raining is an important and highly challenging problem. To address this problem, some depth or density guided single-image de-raining methods have been developed with encouraging performance. However, these methods individually use the depth or the density to guide the network to conduct image de-raining. In this paper, a noveljoint depth and density guided de-raining(JDDGD) method is technically developed. The JDDGD starts with adepth-density inference network(DDINet) to extract the depth and density information from an input rainy image, followed by adepth-density-basedconditional generative adversarial network (DD-CGAN) to exploit the depth and density information provided by DDINet to achieve adaptive rain streak and fog removal. To prevent the spatially-varying local artifacts, an effectiveglobal-local discriminatorsstructure is introduced in the proposed DD-CGAN to globally and locally inspect the generated images. In addition, multiple loss functions includingmulti-scale pixel loss,multi-scale perceptual loss, andglobal-local generative adversarial lossare also jointly used to train our model to achieve the best performance. Both quantitative and qualitative results show that the proposed JDDGD method achieves superior performance than previousnon-guided,density-guided, anddepth-guided de-rainingmethods.