Optical and SAR Image Registration with Deep Reinforcement Learning
Rui Liu, Hongsheng Zhang · 2024
The rise of multimodal big-earth data and deep learning in recent years has promoted the development of multimodal image registration. Registration of optical and synthetic aperture radar (SAR) is significantly enhanced by the geometrical differences between the two modalities, apart from the unimodal image registration. Despite the widespread use of deep learning, however, some shadows still exist in this direction, such as insufficient training data and mismatching in local regions. In this work, the potential of reinforcement learning is explored in optical and SAR image registration. Specifically, reinforcement learning explores the possible displacement directions and magnitudes of the four predefined corner points in a specific search space, to solve the affine or homograph matrix based on the displacements of the corner points, and finally obtain the alignment results of different modes. Results show that with reinforcement learning, there is improvements in the visualization of image registration.