Multi-perspective SAR to 3D Translation using Generative AI
Michael Newey, James T. Kuczynski, Rebecca M. Laher, Michael T. Chan, Alexandru N. Vasile · 2024
This work explores the use of generative adversarial networks (GAN) for multi-look SAR to 3D conversion. We extend 2D-to-2D image translation techniques such as CycleGAN to convert SAR imagery to 3D, taking advantage of existing LiDAR data to provide the 3D information for model training. We use collected X-band radar data from the MITLL ARTB sensor, LiDAR from the MITLL AOSTB sensor and USGS public data in our experiments. We evaluate GAN-based translation performance on large sub-urban scenes as well as on small chips centered on ground vehicles. We evaluate the performance of the algorithms with different number and extents of synthetic aperture radar look angles. Finally, for the case of under- or non-represented cases in training data, we introduce a novel inverted simulation augmentation training-and-test procedure for target classification.