Stakeholders and their respective costs-benefits in IR evaluation
Cécile L. Paris, Nathalie Colineau, Ross G. Wilkinson · 2009
Evaluations in Information Retrieval are dominated by measures of precision and recall. Is that enough? Probably not, as it somewhat assumes that all information seeking tasks are equal, and that everyone needs the same thing. In this position paper, we advocate a consumers ’ guide to systems that aim at supporting information seeking tasks. We propose a method that provides guidance in whole-of-system evaluations, explicitly considering all participants and both sides of the “bang for buck ” equation. 1.