MapReduce for HITS Algorithm with Application to Chinese Word Networks
Qingxian Wang, Li Li · Journal of Convergence Information Technology · 2011
This paper presents a design and implement of HITS algorithm on Hadoop with application to Chinese words networks analysis. To access the importance of word in the text, we build a directed Chinese word network based on word adjacency information, and similar to the idea of HITS algorithm we define values of authority and hub for each word associated with in-degree and outdegree. In order to solve the problem of large-scale computation requirements, we design an algorithm based on Hadoop platform with map/reduce framework. In this implement, a division unit is a record composed of a word and its adjacency information, and the values of authority and hub of words are calculated in parallel by Map and Reduce. Experiment results shows that the implement is effective.