Assistive image caption and tweet development using deep learning
Parth Birthare, R. Maheswari, R. Ganesan, P. Vijaya · Institution of Engineering and Technology eBooks · 2023
In this rapidly growing world and with the technological boom, there are a huge variety of applications and devices that generate an enormous amount of data every second. With unstructured data being the most difficult to manage and keep track of, there has been a drastic increase in the amount of visual data generation. To keep track of such data for further insights and use, a textual descriptor is often needed and getting it is the primitive step for any analytics. A manual description is subjective and not appropriate for larger data. This issue is addressed by automation, hence opening gates for computer vision and artificial intelligence in the domain. Another area that has changed multimedia communication and has seen a great deal of advancement is social media. Applications like Twitter have become an indispensable part of people's lives. Moreover, Progressive Web Application (PWA) is a term that has started to be implemented in various applications. It gives an on-par experience with native apps and has become more prevalent. This work Assistive Image Caption and Tweet (AICT) aims to set a new horizon by combining these applications, and setting a base for future applications and devices. It does so by using deep learning techniques such as convolution neural networks (CNN) and Long Short-Term Memory (LSTM) to generate captions for images within milliseconds, natural language processing (NLP) to generate the text in different languages along with the audio to assist visually impaired people, and an automated assistive tweet function that directly tweets the image with its caption in the language desired.