Implementing of Massive Log Analysis System Based on Parallel Computing
WU Jian-guo · Computer Technology and Development · 2013
On the basis of analyzing log type and features deeply,design and implement a massive log processing system based on parallel computing.It adopts the method of cluster to collect log in parallel way,store in the distributed file system,and analyze log by parallel computing.The system achieves log collection and analysis through automated processing,can effectively carry on security maintenance,system performance optimization,system failure check after the system deployment.The system combines distributed and cloud computing solutions to improve efficiency of log processing,solves the major problems of massive logs processing effectively,provides a complete and effective solutions for massive logs processing.