Benchmarking Image Retrieval Applications
Henning Müller, Antoine Geissbühler, Stéphane Marchand‐Maillet, Paul David Clough · 2004
Content–based visual information retrieval is an important research topic in the computer vision field sind the early 1990s. A large number of systems have been developed as research prototypes as well as commercial and open source systems. Still, there has not been a general breakthrough in performance yet and important real–world application stay fairly rare. The very large amount of available multimedia information creates a need to develop new tools to explore and retrieve within mixed media databases. The replacement of analog films by digital consumer cameras and the increasing digitisation in several fields such as medicine will still increase this need. One of the reasons for the impossibility to show an increase in performance is the simple fact that there is no standard for evaluating the performance of systems. In the last years a rising number of proposals have been made on how to evaluate or not to evaluate the performance of visual information retrieval systems which underlines the importance of the issue. Several benchmarking events such as the Benchathlon, TRECVID and imageCLEF have been started, with varying success. This article described mainly the work of the University of Geneva on benchmarking of visual information retrieval systems. A special emphasis will be on the Benchathlon and imageCLEF evaluation events and their methodology and outcome. 1.