A probabilistic similarity framework for content-based image retrieval
Selim Aksoy, Robert M. Haralick · 2001
Content-based retrieval from image databases has become a popular research area where conventional database retrieval methods are not sufficient because they depend on exact matches of keywords and require an enormous amount of human involvement during manual annotation. Initial work on content-based retrieval focused on using low-level features like color and texture for image representation, and a geometric framework of distances in the feature space for similarity. A challenging problem in image retrieval is the fusion of information from multiple features and similarity measures. In this dissertation, we pose the retrieval problem in a probabilistic framework where the goal is to minimize the classification error in a setting of two classes; the relevance and irrelevance classes of the query. We propose effective solutions to different levels of the retrieval process within this framework. Feature extraction and normalization is done by maximizing class separability, similarity is measured using likelihood and posterior ratios, and post-processing is done using graph-theoretic image grouping and a Bayesian relevance feedback architecture. A key aspect of our framework is a two-level modeling of probability. The first level uses parametric density models to compute class-conditional probabilities from feature vectors and can be interpreted as a mapping from the high-dimensional feature space to the two-dimensional probability space. The second level includes training simple linear classifiers in multiple probability spaces for multiple feature vectors and corresponds to a modeling of “probability of probability” to compensate for errors due to imperfect density modeling in the feature space. Furthermore, classifier combination rules and a naive Bayesian network effectively fuse information from multiple features and similarity models. Performance evaluation was done using extensive experiments on three groundtruth databases including aerial, satellite, texture and stock photo images. The proposed probabilistic framework performed more robustly and significantly better than the commonly used geometric framework and two competing algorithms from the literature. We obtained 8–20% relative improvement in precision over the cases where the best feature vectors were used individually. Moreover, a few feedback iterations resulted in an average precision of more than 94% for all three databases.