Analysis of user attitude and behaviour in evaluating a personalized search engine
Effie Lai‐Chong Law, Tomaž Klobučar, Matic Pipan · 2006
This paper reports an empirical work on user-based relevance evaluation of a personalized search engine (PSE). The aim of the work is threefold: To develop metrics for evaluating PSE; to study how users' trust in personalized information retrieval systems influences their relevance judgments; to identify patterns of relevance criteria applications. Our findings corroborate some of those of the previous work and reveal some new phenomena: the optimality of sample size, consistency of relevance criteria application and role of pre-formed trust in automation.