AN EFFICIENT APPROACH TO REDUCING MEMORY CONSUMPTION FOR STORING THE LARGE NUMBER OF SMALL SIZE FILES IN HDFS
Akash Saxena, RajeshKumar Rameshbhai Savaliya · Journal of Critical Reviews · 2020
Hadoop is the open Source framework of apache and Hadoop framework is developedfor work with Big-Data. It is also specially design for distributed data management.It is used to data storing and accessing data with respect to large size data files. Hadoop Distributed File System (HDFS) used to store the large size data files in the cluster environment of computer system. Hadoop software implementations and framework which has two different components MAPREDUCE and HDFS. Now day HDFS became very popular to manage and handle large size files with very high performance. But the working with the large number of small size files becomes current challenged in the HDFS. Large number of small files impose more burdens of memory consumption, creationas well as management of metadata on NameNode [10].This paper present on propose approach and analysis for memory consumption to existing HDFS and propose framework to enhance the memory management for processing large numbers of small size files in HDFS.