Measuring Web Search Effectiveness: Rutgers at Interactive TREC
Nicholas J. Belkin, Gheorghe Mureşan · 2004
The purpose of this paper is multi-fold. We discuss methodologies and measures of effectiveness that, in our experience, mainly in the TREC Interactive track, have proven successful in painting an accurate picture of the user interaction when seeking information on the Web. We classify the measures and discuss the context s when they can be used. We attempt to provide guidelines as to which measures are appropriate in certain conditions. Measures of effectiveness Evaluation methods were not born with or created solely for Information Retrieval. The field of IR borrowed methodologies of scientific investigation from other areas of research, including experimental design, statistical analysis and a range of qualitative and quantitative measures, and also created its own, targeted at the specific tasks that it investigates. Effectiveness of a retrieval system is an ectosystem measure, related to the user’s satisfaction with the system output and the interaction outcome (Korfhage, 1997). Modern IR systems are complex systems expected to