Prediction of the Visual Similarity Between Photos using Several Pre-Traind CNNs
Hiroshi Omori, Kazunori Hanyu, Masako Yamashita, Satoshi Shimada · 2018
We have measured the visual similarity between photos manually by making dozens of people classify similar photos. We proposed the method how to predict the unmeasured visual similarity between photos by making use of several 1000 class classifications to photos by pre-trained Convolutional Neural Networks (CNNs). We conducted a survey asking students to take photos of their lives that interest them. 200 photos were collected from 2012 to 2014. The visual similarity matrix P between these 200 photos was measured. The similarity matrix Q between them was easily obtained by CNNs. Divide 200 photos into two groups A and B. Assuming that only the visual similarity between photos included in A was measured. Minimizing Kullback - Leibler divergence between two Gaussian distributions with mean 0, covariance matrices P and Q, P could be restored from the visual similarity within A and Q. It was shown that restoration of P went quite well, if the photos included in A were similar to the photos included in B. On the other hand, it was shown that restoration of P went fairly well, otherwise.