Generative Adversarial Networks for Hard Negative Mining in CNN-Based SAR-Optical Image Matching
Lloyd Haydn Hughes, Michael Schmitt, Xiao Xiang Zhu · 2018
In this paper we propose a deep generative framework, based on a generative adversarial network (GAN) and an auto encoder (AE), for generating non-corresponding SAR patches to be used in hard negative mining in situations of limited data quantities. We evaluate the effectiveness of this formulation of hard negative mining for reducing the false positive rate (FPR) and improving network determinability in a SAR-optical patch matching application. Our generative network is trained to generate realistic SAR images using an existing SAR-optical matching dataset. These generated images are then used as non-corresponding, hard negative samples for training a SAR-optical matching network. Our results show that we are able to generate realistic SAR images which exhibit many SAR-like features, such as layover and speckle. We further show that by fine tuning the original matching network using these hard negative samples we are able to improve the overall performance of the original SAR-optical matching network.