An Evaluation Method of Web Search Engines Based on Users' Sense
Takashi Ohtsuka, Koji Eguchi, Hayato Yamana · NTCIR · 2004
It is not easy to obtain the useful information from the Web space because the space is increasing and large. Therefore, the Web search system becomes indispensable and the improvement of its performance is required. Then, the researches on search engine’s evaluation methods are being important. The evaluation based on the conventional evaluation methods, however, is not always equal to the user’s evaluation. The reason is most of the conventional evaluation methods are based on precision and recall, which do not take users’ sense for search engines into consideration. In this paper, we propose “a user oriented evaluation criterion” that evaluates the performance of Web search systems by considering users’ actions when they retrieve Web pages. We also evaluate the proposed criterion in comparison with the conventional methods by measuring the time spent on search as the users’ satisfaction degree.