ATTENTION BASED IMAGE CAPTIONING USING DEEP LEARNING
Tejaswini Nakirekanti, D. Deepika · International Journal of Innovative Research in Advanced Engineering · 2021
The process of deriving written descriptions from a picture based on the items and behaviours portrayed is known as image caption generation. The resulting description will represent what is in the image and the interactions among the objects. However, like with any other image processing challenge, recreating this characteristic in an artificial system is a difficult feat, and so the adoption Deep Learning procedures can help to tackle the problem. The main goal of this project is to generate an image captioning model that can provide a concise and relatable description of the picture by incorporating an attention mechanism. Ideally, the model can be separated into two components – one is called Encoder and the other is a Decoder. The encoder used here is a Google InceptionV3 pre-trained model. And the decoder is a language-based model, Gated Recurrent Unit (GRU) to translate the characteristics and objects provided by the encoder into natural sentences. Furthermore, the model's effectiveness is aided by the attention mechanism. To conclude, the model is trained on a subset of the MS COCO dataset, which contains about 80,000 pictures with at least five descriptions each and evaluated based on the BLEU score. To reduce the loss, the model weights are tuned using the categorical cross-entropy loss function. The results of the model are promising and competitive.