Research on Hadoop-based Network Log Analysis System
Lixin Ke · Computer Knowledge and Technology · 2010
Viewing Web log is an important measure to investigate system failures and monitor operating status of the system.Administrators can examine what happened at a certain time,they can also analyze each log file for purpose of acquiring knowledge.If the administrator only review the status by virtue of checking log files,it is difficult to find useful information that contained in those files,as the log files is characterized by large volume of data and uneasy to read.Distributed computing technology can be used to solve this problem exactly.Process of Syslog log collection is explained,details of the Hadoop distributed computing framework is introduced in this paper,and a Hadoop-based network log analysis system is also designed and implemented.Experimental results show that the system is effective and practical.