VRFF: Video Registration and Fusion Framework
Meng Sang, Housheng Xie, Yang Yang · 2024
Through the process of infrared and visible image registration and fusion, we can generate a composite image that encapsulates the features of both infrared and visible images, thereby enhancing decision-making capabilities in subsequent advanced visual tasks. Despite significant strides made in image registration and fusion research, a noticeable gap remains in the field of video registration and fusion. The direct application of image registration and fusion techniques to videos often results in a flickering effect. To address this issue, we redefine the workflow of the image registration and fusion framework (IRFF), which includes stages of matching key points (MKPs) extraction, image alignment, and image fusion. Based on this workflow, we propose a new video registration and fusion framework (VRFF). In the image alignment stage, we propose the integrated previous frames (IPF) strategy and employ the Moment algorithm, both of which are based on temporal relationships. Additionally, we adopt a new strategy to retrain the MKPs extraction network and redesign the image fusion network for enhanced performance. Experimental results demonstrate that the VRFF exhibits superior performance on video streams. Furthermore, we also explore the effectiveness of fused images in advanced visual applications.