Weblog Mining Based on Hadoop
Chen Hua-pingb · Jisuanji gongcheng · 2011
The mass data from Web are distributed,heterogeneous and dynamic,so the current data mining system based on single node has developed to a bottleneck.Using the advantage of cloud computing——distributed processing and virtualization,this paper presents a Weblog analysis platform under the Hadoop's cluster framework based on cloud computing,it also presents a hybrid algorithm which can distributed process in the cloud computing environment.To further verify the effectiveness and efficiency of the platform,it uses the improved algorithm to mine users' preferred access path in Weblog on the platform.Experimental results show that,using distributed algorithm to process large number of Weblog files in the cluster,can significantly improve the efficiency of Web data mining.