An Attempt at Zero-shot Ancient Documents Restoration Based on Diffusion Models
Hayata Kaneko, Yuuya Yoshizu, Ryuto Ishibashi, Lin Meng · 2023
The virtual restoration of ancient documents using deep learning is an emergency and an expected work. However, GANs-based image-to-image translation approaches hit the degradation data shortage, a hardness to build one-to-many restoration models, and a limitation for large deformation. In this study, we apply zero-shot restoration based on Diffusion models to ancient degraded documents, specifically, leverage inpainting of Denoing Diffusion Restoration Models (DDRM) for missing ancient characters. Furthermore, we introduce a noise masking method, which limits the attention area of predicted noise images in the reverse process. Noise masking forces DDRM to generate faithful objects following mask images, so that has high usability without re-training of deep neural networks. The zero-shot restoration and noise masking prompt GUI-connecting restoration of missing characters, leading to realizing a cooperative application with humans for ancient document restoration.