The Improved Pagerank in Web Crawler
Zhang Ling, Qin Zheng · 2009
Pagerank is an algorithm for rating web pages. It introduces the relationship of citation in academic papers to evaluate the web page's authority. It gives the same weight to all edges and ignores the relevancy of web pages to the topic, resulting in a problem of topic-drift. On the analysis of several pagerank algorithms, an improved pagerank based upon thematic segments is proposed. In this algorithm, a web page is divided into several blocks by Html document's structure and the most weight is given to linkages in the block that is most relevant to given topic. Moreover, the visited outlinks are regarded as feedback to modify blocks' relevancy. The experiment on Web crawler shows that the new algorithm has some effect on resolving the problem of topic-drift.