Image Captioning on Fine Art Paintings via Virtual Paintings
Yue Lu, Chao Guo, Xingyuan Dai, Fei–Yue Wang · 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI) · 2021
Machine learning in fine art paintings is attracting increasing attention recently. Image captioning of paintings is of great importance for painting analysis, but it is rarely studied. The paintings have abstract expressions and lack annotated datasets, leading to the data-hungry problem in painting captioning. Thus, painting captioning has more significant challenges than photographic image captioning. This paper makes a novel attempt at generating content descriptions of paintings. We generate virtual paintings using the style transfer technique to deal with the data-hungry problem, then train the painting captioning model via a two-step manner. We evaluate our method on an annotated small-scale painting captioning dataset and demonstrate our improvements.