HDFS memory usage analysis
B Purnachandra Rao, N. Naga Malleswara Rao · 2017 International Conference on Inventive Computing and Informatics (ICICI) · 2017
Due to high use of online activities, there will be lot of data which is getting generated. To handle this data there should be an efficient system which can process the data effectively. One such system is Hadoop Distributed File System. HDFS consists of number of nodes, and one is the master among them while all others are slave nodes. Master node is called as NameNode and slave nodes are called as DataNodes. System will record the activities in the log file. Whatever the transactions are happening at the front end everything will get recorded at backend. We can capture this data and analyze to find out the system behavior in the failure situation. Since we are using the system for large transactions we need to have an idea on usage of the resources in the HDFS such as memory usage. Analyzers can get the stats of the operating system and the other devices separately i.e, for each service we need to have one analyzer to collect the stats. This paper presents the analyzer or shipper for collecting the metrics from operating system, memory and the other installed services.