Evaluating Intelligent Search Agents in a Controlled Environment Using Complex Queries: An Empirical Study
Nurul I. Sarkar, Xiumin Wang, Shan Liang, Bo Hong, Jun Li · International Journal of Database Theory and Application · 2017
There is a growing interest in using intelligent search agents (ISAs) in e-commerce and online businesses worldwide in recent years.This interest results from the availability of various sophisticated and powerful intelligent agents that can automate the process of searching through and evaluating reams of information on the Web.While efforts have been made recently to develop various powerful ISAs and multi-agent systems, very little is understood and known about evaluation of such agents.In this paper we describe a simple experimental setup ('system') that can be used to evaluate ISAs without using any complex algorithm and mathematical analysis.The idea is to evaluate search agents based on a performance metric in a controlled environment using complex search queries.For an efficient evaluation of ISAs, we introduce a new metric called 'search speed' which is a ratio of the number of results returned per second per query to the round trip time.This paper provides an in-depth performance comparison of four selected ISAs (Copernic, FirstStop Websearch, iMeta, and WebFerret).These search agents were selected based on their availability, popularity, and interesting characteristics and features.The analysis and empirical results reported in this paper provide some insights into the evaluation of ISAs which may help researchers to evaluate similar search agents.