Evaluating and Enhancing Contextual Search with Semantic Similarity Data
Ana Gabriela Maguitman, Carlos M. Lorenzetti, Rocío L. Cecchini · IGI Global eBooks · 2011
Performance evaluation plays a crucial role in the development and improvement of search systems in general and context-based systems in particular. In order to evaluate search systems, test collections are needed. These test collections typically involve a corpus of documents, a set of queries and a series of relevance assessments. In traditional approaches users or hired evaluators provide manual assessments of relevance. However this is difficult and expensive, and does not scale with the complexity and heterogeneity of available digital information. This chapter proposes a semantic evaluation framework that takes advantages of topic ontologies and semantic similarity data derived from these ontologies. The structure and content of the Open Directory Project topic ontology is used to derive semantic relations among a massive number of topics and to implement classical and ad hoc retrieval performance evaluation metrics. In addition, this chapter describes an incremental method for context-based retrieval, which is based on the notions of topic descriptors and topic discriminators. The incremental context-based retrieval method is used to illustrate the application of the proposed semantic evaluation framework. Finally, the chapter discusses the advantages of applying the proposed framework.