Prediction of the Visual Similarity between Photos Making Use of Deep Learning
Hiroshi Omori, Kazunori Hanyu, Satoshi Shimada · Transactions of Japan Society of Kansei Engineering · 2020
By using the objective visual similarity matrix P that is given only to some photo pairs and the image similarity matrix Q that can be calculated automatically for all photo pairs, the visual similarity of the missing parts of P can be predicted. To make Q we use 1000 class classification and semantic segmentation of photos by pre-trained Convolutional Neural Networks (CNNs). Minimizing Kullback-Leibler divergence between two Gaussian distributions with mean 0, covariance matrix P and Q respectively, P can be completed. 200 photos were collected from students at the University of Tokyo and the visual similarity between all photo pairs was measured. 184 photos were also collected from Meiji University students and the visual similarity between photos was not measured. The visual similarity between all 384 photos was predicted by the proposed method.