Benchmarking image and video retrieval
Stéphane Marchand‐Maillet, Marcel Worring · 2006
Multimedia Information Retrieval (IR)techniques and associated systems are now numerous and justify the development of strategies and actions to objectively evaluate their capabilities. A number of initiatives following this line exist, each with its own peculiarities. In this paper, we take a bird's eye view on benchmarking multimedia IR systems (with the particular case of image and video retrieval)and summarize contributions made to a dedicated special session at the ACM Multimedia Information Retrieval Workshop (ACM MIR2006).From the analysis of a classical IR system, we identify locations of interest for evaluation of performance.We review proposals made in the context of existing benchmarks, each specialized in its own aspect and media.