A Novel Architecture to Efficient utilization of Hadoop Distributed File Systems for Small Files
Prem Sagar Sharma · International Journal for Research in Applied Science and Engineering Technology · 2018
Hadoop is known as an open source distributed computing platform and HDFS is defined as Hadoop Distributed FileSystem having powerful data storage capacity therefore suitable for cloud storage system.HDFS was designed for streaming access on large software and it has low storage efficiency for massive small files.For this problem, the HDFS file storage process is improved and therefore files are judged before uploading to HDFS clusters.If the file is of small size then it is merged and its index information is stored in index file in the form of key-value pairs else it will directly go to HDFS Client.Also if all files are processed and no file left to be merged then the merged files go to the HDFS.[7]