A measure for evaluating search engines on the World Wide Web
Teru Agata, Michiko Nozue, 紀彦 服部, Shuichi Ueda · Library and Information Science · 1998
Search engines are a kind of information retrieval systems on the Internet. However, they cannot be evaluated with the measures commonly applied in traditional retrieval experiments, i.e., recall and precision, since recall cannot be calculated in the essentially unrestricted resources of the Internet. The Expected Search Length (ESL) proposed by Cooper in 1968 is a measure for evaluating information retrieval systems and has the possibility of taking the place of recall and precision. The ESL calculates the cost paid by a user, i.e., number of retrieved sites the user must look through before (s)he gets sufficient number of relevant sites. This article proposes a version of the ESL in order to adapt it to evaluate search engines. An experiment was carried out to compare the validity of the ESL with that of precision and recall. Eight search engines of the domestic and overseas were evaluated by precision and ESL. The result of this retrieval experiment shows the adaptability of ESL for the evaluation of the search engines.