Factual or Satisfactory: What Search Results Are Better?
Hong Yu, Jun Lu, Shiqi Zhao · Institutional Repositories DataBase (IRDB) · 2011
An interesting issue "whether the factual search result is better than the satisfactory one?" is discussed in this paper, and all effort is to illustrate the publicly recognized conception "the supreme satisfactory result is optimal" could lead Information Retrieval astray.By contrast, we propose a new hypothesis "the factually result should be the optimal although sometimes unsatisfactory".To implement the pilot study on this, we developed a search-engine (Google) based labeling platform and designed a new evaluation method involving user experiences.The results on the platform show the existence of bias on determining the optimal search results (factual or satisfactory).And the very different performances of NDCG and our evaluation metric demonstrate the weakness of current ranking algorithms in mining and recommending the long-term effective information.