FIRE: Forum for Information Retrieval Evaluation
Mandar Mitra, Prosenjit Majumdar · International Joint Conference on Natural Language Processing · 2008
This talk will present our plans for organizing FIRE, a forum for Information Retrieval evaluation, focused on Indian languages. We will start by reviewing the basic experimental framework and metrics as represented by the Cranfield paradigm. An overview of the main evaluation fora for IR -TREC, CLEF and NTCIR, which are all based on this paradigm, and from which FIRE draws inspiration -will be given. The components of the evaluation framework, viz. corpora, search topics, and relevance judgments will also be discussed.