An Image Comment Method Based on Emotion Capture Module
Qin Li, Jiankai Yin, Yan Wang · 2021 IEEE 3rd International Conference on Frontiers Technology of Information and Computer (ICFTIC) · 2021
The task of generating comments for images has been studied by many scholars. Existing methods, such as sentence generation based on recurrent neural network (RNN), have poor variability due to their structural limitations. In addition, the generated comment is so stiff that lacks the emotional color and value orientation of human language. In order to make the image comment more open and meet the needs of emotion control, we designed a comment generation and style transfer module for pictures. This paper mainly completes the following work: (1) Imitate human emotions by giving positive and negative colors to the resulting sentences. Using the characteristics of generative adversarial network, comments can not only meet the requirement of verisimilitude, but also encourage naturalness and diversity. (2) Design an emotion capture module (ECM) and a more compound loss function. It can complete the emotional reversal of the caption content that needs to be controlled, making it more colloquial and life-like. Experimental results on public datasets and generated dataset show that the proposed model has good effect on style transfer accuracy and content preservation.