Mutual-Feed Learning for Super-Resolution and Object Detection in Degraded Aerial Imagery
Jinze Yang, Kun Fu, Youming Wu, Wenhui Diao, Wei Dai, Xian Sun · IEEE Transactions on Geoscience and Remote Sensing · 2022
The resolution degradation poses a huge challenge for object detection (OD) in the aerial imagery. Existing methods utilize super resolution (SR) based on Generative Adversarial Network (GAN) to restore texture details in degraded images. However, constrained detection results are still acquired due to the object feature difference between restored and clear images. Therefore, we propose a simple-yet-effective learning method called Mutual-Feed Learning (MFL) to solve the problem in this paper. A closed-loop structure is designed via building the feedback connection based on the feedforward connection between the two tasks. It effectively delivers the object spatial and feature information from OD to SR, and provides restoration-enhanced images from SR to OD. Specifically, a Feedback of Region of Interest (FROI) module is introduced to realize a region-level discrimination under the guidance of object information. It guides the discrimination process of super resolution. Furthermore, a Multi-Scale Object Information (MSOI) module is developed to implement a feature-level restoration by narrowing differences in object-related features. It improves the generation process of super resolution. Then object detection can be performed in restoration-enhanced images to obtain more accurate results. Extensive experiments over NWPU VHR-10, COWC, and FAIR1M dataset show that the method can achieve state-of-the-art results.