Evaluating Multi-Model (Metadata-Semantic) Information Retrieval System.
Leyla Zhuhadar, Olfa Nasraoui, Róbert E. Wyatt, Elizabeth Romero Massa · International Conference on Artificial Intelligence · 2009
Abstract—In this paper, we present a variety of methodsto evaluate a multi-model information retrieval system. Theevaluation methods are divided into three steps: (1) evaluatinga metadata driven search engine, (2) evaluating a personalizedcluster-based semantic search engine, and (3) evaluating a dualrepresentation of the semantic user profile for personalized Websearch in an evolving domain. The first part of this paper presentsan overview of the most popular information retrieval models andtheir evaluation methods. The second part is dedicated to thespecific evaluation methods that we used to evaluate each modelof our platform. This platform is already being used by onlinestudents at WKU. As of 2006 the “HyperManyMedia” searchengine has been ranked number 24 on “The Ultimate Guide toUsing Open Courseware 1 ” (between Cambridge University andHarvard Business).Index Terms—Evaluation Methods, Precision, Recall, Informa-tion Retrieval I. I NTRODUCTION What is information? [7] divided information into three cate-gories: information-as-process, information-as-knowledge, andinformation-as-thing. It stated that “information is anythingthat can change person’s knowledge [7].”[16] looked at an information system from the user’sprospective. His framework is human centered, where “theinformation seeker defines the task, controls the interactionwith the search system, examines and extracts relevant in-formation, assesses the progress, and determines when theinformation-seeking is complete [16]”. The framework for thispaper is based on a student-centered approach, where a studentsearches for information, the system retrieves relevant infor-mation to the student’s query, the student uses the informationand, at the end, evaluates its relevance.[14] distinguished between Data and Wisdom. Data isequally available to any user, it is received, stored and retrieved