‘Envision’: Visual Representation System
Sheetal Jaybhay, Dhruvi Jain, Siddhi Bhagat, Priya Chaudhari, Vanessa Dbritto · 2020
The use of technology is increasing day by day in most of the possible areas. Education is one of the area in which technology has helped students in understanding the basic concepts and theories related to their studies. It has been observed that the primary students face problem in understanding some of basic concepts like arithmetic word problems. Envision is a system that incorporates natural language text input to construct scenes in real-time. Such type of system forges the way for systems in which interaction with and construction of graphical media can be as natural as communicating via text. System aims at taking the basic arithmetic word problems of primary section as input and depicting a solution as an image on the output screen. Natural Language Processing (NLP) is used to process the text given by user and to find the objects to create the scene. The image dataset used in this process will grow as the new objects are encountered in the word problem. The system produces photo-realistic images based on the input text using Generative Adversarial Networks (GAN) and helps students to realize what exactly the sentence means. Low resolution images are formed using Stage-I GAN by sketching the primal shape and colours of the object based on the given text description. The Stage-II GAN processes Stage-I outcome and generates high resolution images with photo-realistic information. The system will also be useful for teachers and parents to teach their students/children in an effective way.