Large-scale interactive evaluation of multilingual information access systems: the iCLEF Flickr challenge
Paul David Clough, Julio A. Gonzalo, Jussi Karlgren, Emma Barker, Javier Artiles, Víctor Peinado · 2008
Participation in evaluation campaigns for interactive information retrieval systems has received variable success over the years. In this paper we discuss the large-scale interactive evaluation of multilingual information access systems, as part of the Cross-Language Evaluation Forum evaluation campaign. In particular, we describe the evaluation planned for 2008 which is based on interaction with content from Flickr, the popular online photosharing service. The proposed evaluation seeks to reduce entry costs, stimulate user evaluation and encourage greater participation in the interactive track of CLEF. 1. EVALUATION OF IR SYSTEMS Evaluating the performance of Information Retrieval (IR) systems is an important part of the system development process from an engineering point of view, and a crucial part of the