Research of Massive Web Log Data Mining Based on Cloud Computing
Zhen Qi Wang, Hai Long Li · 2013
Internet data is massive, heterogeneous, dynamic, and data is increasingly complex. Through data mining and analysis, we are able to obtain potentially valuable information, but traditional data mining system has bottleneck in data storage and computing power. To solve the problem, by using technology of cloud computing, we design a massive web log data mining and analysis platform based on cloud Hadoop framework. At the same time, in order to improve the efficiency of mining, we realize the parallelization of Apriori algorithm for the massive Web log mining. Then we use the platform to verify the efficiency of Apriori algorithm which has been improved by parallelization.