What happened in CLEF 2008 Introduction to the Working Notes
Carol Peters · CLEF (Working Notes) · 2008
These Working Notes contain descriptions of the experiments conducted within CLEF 2004 – the fifth in a series of annual system evaluation campaigns organised by the Cross-Language Evaluation Forum 1 . The results of the experiments will be presented and discussed in the CLEF 2004 Workshop, 15-17 September, Bath, UK. The final papers - revised and extended as a result of the discussions at the Workshop - together with a comparative analysis of the results will appear in the CLEF 2004 Proceedings, to be published by Springer in their Lecture Notes for Computer Science series. CLEF organises a series of evaluation tracks designed to test different aspects of mono- and cross-language information retrieval system development with the main focus on European languages. The objective is to provide an infrastructure that facilitates experimentation with all kinds of multilingual information access – from the development of procedures for monolingual retrieval operating on many languages to the implementation of complete multilingual multimedia search services. In a ddition, CLEF aims at encouraging contacts between the R&D and the application communities and promoting the industrial take-up of research results. The main features of the 2004 campaign are briefly outlined here below in order to provide the necessary background to the experiments reported in this volume. 1. Tracks and Tasks in CLEF 2004 In recent years, CLEF has distinguished between core tracks, which were those offered regularly in each campaign (the monolingual, bilingual, multilingual and domain-specific tracks), and additional tracks, which were organised on an experimental basis with the objective of identifying new requirements and appropriate methodologies for their testing in a cross-language context. This distinctio n no longer held in 2004. The great success of the so-called additional tracks in CLEF 2003, and in particular of th e tracks that tested systems for question answering and image retrieval, has led to their inclusion as regular tracks this year. This has meant that CLEF 2004 marks a breaking point with respect to the previous campaigns. The focus is no longer on multilingual document retrieval but has diversified to include different kinds of text retrieval across languages (from documents to exact answers) and retrieval on different kinds of media (not just text but image and speech as well). CLEF 2004 thus offered six tracks designed to evaluate the performance of systems for: • mono-, bi- and multilingual document retrieval on news collections (Ad-hoc) • mono- and cross-language domain-specific retrieval (GIRT) • interactive cross-language retrieval (iCLEF) • multiple language question answering (QA@CLEF) • cross-language retrieval on image collections (ImageCLEF) • cross-language spoken document retrieval (CL-SDR)