Design and Implementation of Hadoop's Log Statistics Analysis System Based on Mahout
Zhen Cao · Computer engineering & Software · 2015
With the wild use of Hadoop and appearance of Hadoop Yarn,the scale of clusters is getting larger.And the open source implementation of the clusters operating state monitor system in the Hadoop ecosystem has been very mature,but there is not yet a platform for statistical analysis of the running trend of the Hadoop Jobs.In this paper,a job resource statistical analysis platform Yarn Hadoop is presented,which is for the Cluster Administrator and the ordinary users,with the double dimension of date and user to analyze the job,and get the standard of Hadoop operation.