FATO: Frequency Attention Transformer for Omnidirectional Image Super-Resolution
Hongyu An, Xinfeng Zhang, Shijie Zhao, Li Zhang · 2024
Benefiting from the 360 • field of view (FoV) of the omnidirectional images (ODIs), users could enjoy an immersive experience with head-mounted devices or computers.High-resolution ODIs can provide pleasing visual experience and boost the performance of related visual tasks.Therefore, Super-resolution (SR) is an essential technique during the application of ODIs.However, traditional SR methods fail to enhance the most widely utilized equirectangular projection (ERP) format ODIs due to projection distortions.Existing ODI-SR methods take the latitude-related position information as a prior, but lack the adaptation to the ERP content distribution characteristics.To address this issue, we propose a novel Frequency Attention Transformer ODI-SR (FATO) network focusing on highfrequency details of ODIs.In particular, we transform an ODI into fine-grained patches in the frequency domain through Discrete Cosine Transform (DCT).After that, we design a frequency selfattention mechanism to capture the relationship between different frequency patches.Subsequently, we introduce a frequency loss function to further constrain the network.Extensive experimental results demonstrate that the proposed FATO achieves superior performance over state-of-the-art methods on ODIs.