Performance Analysis of an Ontology Based Crawler Operating in a Distributed Environment
Wael A. Gab-Allah, Ben Bella S. Tawfik, Hamed Nassar · Zenodo (CERN European Organization for Nuclear Research) · 2017
Crawlers are being increasingly utilized to retrieve information from distributed information sources, such as the Web. We have implemented one that makes use of some novel algorithms and techniques, namely, a novel IR architecture, an efficient query expansion algorithm based on WordNet, a new crawling technique based on ontology and a new rapid filtering algorithm based on semantic similarity. The experimental results of the implemented crawler, named Ontology Based Distributed Information Retrieval (OBDIR) system, show superiority to those obtained from systems based on the standard Breadth First (BF) search technique. In this paper we analyze the performance of the OBDIR system. We develop a probabilistic model that captures the operational dimensions of the system. The model makes heavy use of Bayes' theorem and can help establish a foundational theory for DIR. We study such performance metrics as recall and precision, and allude to other performance tools such as accuracy and ROC space. The study shows that by carefully choosing the keywords the performance of the crawler is enhanced greatly.