Multilingual interactive experiments with Flickr
Paul David Clough, Julio A. Gonzalo, Jussi Karlgren · 2006
This paper presents a proposal for iCLEF 2006, the interactive track of the CLEF cross-language evaluation campaign. In the past, iCLEF has addressed applications such as information retrieval and question answering. However, for 2006 the focus has turned to text-based image retrieval from Flickr. We describe Flickr, the challenges this kind of collection presents to cross-language researchers, and suggest initial iCLEF tasks. 1 Information Retrieval Evaluation by User Experiment Information retrieval systems, especially text retrieval systems, have benefited greatly from a fairly strict and straight-laced evaluation scheme, which enables system designers to run tests on versions of their system using a test collection of pre-assessed data. These relevance-oriented experiments shed light on comparative system performance and enable both introduction of new algorithms and incremental optimization. However, batch-oriented system evaluation based on large amounts of data, abstracted away from situational constraints, variation in usage, and interactiveness issues only addresses some of the bottlenecks to build a successful system. The CLEF1 Interactive Track (iCLEF2) is devoted to the comparative study of user inclusive cross-language search strategies. Over the past 5 years, iCLEF has studied three cross-language search tasks: retrieval of documents, answers and