Historical Postcards Retrieval through Vision Foundation Models
Anis Amri, Salvatore Tabbone · 2024
The analysis of historical documents presents challenges in automated processing, given issues such as degradation over time and the need to extract information from a large data source. This paper proposes a methodology, comprising two key stages: text extraction using Optical Character Recognition (OCR) and Vision Foundation Models (VFM) for the retrieval process within a challenging collection of 4,294 historical postcards from the East of France region. This approach allows users to effortlessly find postcards that align with their interests and preferences. VFMs and the textual information extracted from the postcards play a key role in this process, by providing a robust and efficient way to match user queries to relevant postcards in the dataset.VFMs trained on large datasets offer a solution to reduce dependence on annotated data and enhance model versatility. For our retrieval stage, we have selected two VFMs, CLIP and DINOv2, and evaluate their performance using quantitative metrics to identify the model yielding the best results.