Entropy-based link analysis algorithm for web structure mining
Jianfu Li · Jisuanji gongcheng yu sheji · 2006
In Web structure mining,hyperlink induced topics search(HITS) algorithm has been widely employed to analyze authorities and hubs of pages returned by search engine.However,except for useful information,most of content sites contain some irrelevant hy-perlinks,such as advertisements and navigation panels.And because of these extra hyperlinks,HITS is found insufficient in analyzing advertisement or irrelevantpages,which would result in high authority values or hub values for these pages.In order to solve this problem,Shannon information entropy is introduced to HITS algorithm,thus the entropy-based link analysis algorithm is presented to mine Web informative structures.The key idea of this algorithm is to utilize shannon information entropy to represent the knowledge hided in link texts.