Story visualization using image-text matching architecture for digital storytelling
Arian Yturrizaga-Aguirre, Camilo Silva-Olivares, Willy Ugarte · 2022
Currently, the techniques for generating images from text used to visualize stories have serious limitations in terms of image quality, which prevents quantifying their impact in real life scenarios. An example of this occurs in the field of education, where digital storytelling is used as a tool to incite teaching. For this reason, we propose to design a web interface that allows primary school children to write a short story and obtain, as a result, a sequence of coherent and representative images of said content, emulating a conventional process of educational digital storytelling. We describe the use of an Image-text matching architecture based on NLP and Image Retrieval for the story visualization task focused on digital storytelling. To evaluate the performance of the architecture, the quantitative metrics: WuPalmer and cosine similarity were used, in addition to qualitative metrics.