Research on Topical Crawler of Shark-Search Algorithm and Hits Algorithm
Qingxiu Wu · Computer Technology and Development · 2010
Topical crawler is the core technology to achieve vertical search engine.There are two important crawling algorithms to be introduced:content-based evaluation of Shark-Search algorithm and link-based relationships Hits algorithms.It analyzed their respective advantages and disadvantages and proposed a new topical crawling strategy that is to combine the two algorithms which include content-based evaluation and link-based relationships,to judge whether url to be downloaded is good or bad,and implements a topical crawler.This new crawling strategy can make up for the deficiencies of the two algorithms.With the Shark-Search algorithm and the algorithm of the Hits contrast,it is inferred that the effect of using the new topical crawling algorithm which reaches the degree of accuracy is better than those two algorithms.