Crowdsourced Evaluation of Personalization and Diversi- fication Techniques in Web Search
David Vallet · 2011
services have been proven to be a valuable re- source for search system evaluation or creation of test collections. However, there are still no clear protocols to perform a user- centered evaluation of approaches that consider user factors, such as personalization or diversification of results. In this work we present two complementary evaluation methodologies to evaluate personalization and/or diversification techniques. Our preliminary results are encouraging, as they are consistent with previous ex- perimental results. This leads us to believe that these evaluation methodologies could allow performing inexpensive user-centered evaluations. As a result of this work, a test collection for personal- ized and diversification approaches has been made available.