Semantic Image Captioning using Cosine Similarity Ranking with Semantic Search
Kirti Jain, Shrey Gandhi, Shivam Singhal, Suryansh Rajput · 2023
Social media has become an integral part of our daily lives, and its use has increased exponentially in recent years. With the rise of smartphones and internet accessibility, people of all ages, ranging from school-going children to older generations, are using it to stay connected with friends, family, and the world around them. The diverse range of social media platforms available today, such as Facebook, Twitter, and Instagram, provides users with a variety of options to express themselves and share their interests and experiences with others. However, many users often struggle with finding meaningful captions for their images, and browsing multiple sites to find a suitable caption is a tedious task. In such cases, an image captioning system is a valuable tool for social media users which can help overcome the challenges faced by the social media users and make their social media experience much more enjoyable and productive. In this research paper, we develop an image captioning system that utilizes a database of captions to generate relevant captions for input images. For this, we first convert the input image into a vector using a pre- trained deep learning model which uses Vision Transform (ViT) as the VAE encoder, and then perform semantic search to find the most similar cosine vector in the caption database. Finally, we use the corresponding caption associated with the most similar vector as the output caption for the input image.