Improving Image Reconstruction and Synthesis by Balancing the Optimization from Frequency Perspective
Xuan Dang, Guolong Wang, Xun Wu, Zheng Qin · 2024
Image reconstruction and synthesis tasks have been boosted by deep learning technologies in recent years. However, most existing deep learning-based transformation methods utilize loss functions in the spatial domain to guide the training process and having unavoidable defects such as image blurring, and checkerboard artifacts in certain cases. In order to improve the quality of generated images, we analyse the training process from frequency perspective, unveil the inherent optimization imbalance between low and high frequency components theoretically, and based upon which we propose the Gradient Balanced Frequency Loss (GBFL) to mitigate the imbalance by taking into account the historical accumulated gradients of each frequency component and reweighting the optimization process accordingly. Extensive quantitative and qualitative experiments demonstrate the versatility and efficiency of our proposed GBFL, as well as robustness to out-of-distribution test samples.