CorAFR: Arbitrary-time frame reconstruction from motion-blurred and correlation images
Pan Wang, Toru Kurihara, Jun Yu · Neurocomputing · 2025
This paper proposes a deep-learning solution based on a three-phase correlation image sensor (3PCIS) to recover consecutive sharp frames from a single motion-blurred image at any given moment within the exposure time. The correlation image generated by 3PCIS is the temporal correlation between incident light and reference signals, carrying motion information about the speed and direction of moving objects during the exposure time. This is crucial for restoring clear texture details. Therefore, we combine correlation images and motion-blurred images as inputs, and build a two-stream network with temporal encoding to restore sharp frames at any given moment. Two key designs in our model are 1) a Double-gated Feature Fusion (DGFF), facilitating effective fusion of complementary information for high-quality outputs; 2) a Gaussian-based Temporal Encoding (GBTE), transforming arbitrary time within exposure time into feature-like representations for establishing correspondence between time and restored frames. Our model can produce any number of sharp frames by changing time. Experimental results demonstrate that our solution significantly outperforms the state-of-the-art methods on public datasets and real-world scenarios.