A Simple Focused Crawler.
Ah Chung Tsoi, Daniele Frosali, Marco Gori, Markus Hagenbuchner, Franco Scarselli · Use Siena air (University of Siena) · 2003
A focused crawler may be described as a crawler which returns relevant web pages on a given topic in traversing the web. There are a number of issues related to existing focused crawlers, in particular the ability to ``tunnel'' through lowly ranked pages in the search path to highly ranked pages related to a topic which might re-occur further down the search path. We will introduce a simple focused crawler, which is described by two parameters, viz., degree of relatedness, and depth. Both provide an opportunity for the crawler to ``tunnel'' through lowly ranked pages. Results from initial experiments are promising and motivate for further research.