Multimodal Neural Networks in the Problem of Captioning Images in Newspapers
Patryk Kaszuba · Annals of Computer Science and Information Systems · 2023
This paper presents the effectiveness of different multimodal neural networks in captioning newspaper scan images.These methods were evaluated on a dataset created for the Temporal Image Caption Retrieval Competition, which is a part of the FedCSIS 2023 conference.The task was to predict a relevant caption for a picture taken from a newspaper, chosen from a given list of captions.The results we obtained show the promising potential of image captioning using CLIP architectures and emphasize the importance of developing new multimodal methods for problems that combine multiple disciplines, such as computer vision with natural language processing.