A Novel Data Encryption in HDFS
Thanh Cuong Nguyen, Wenfeng Shen, Jiwei Jiang, Weimin Xu · 2013
Hadoop distributed file system (HDFS) is an open source software which can be used to build cloud storage system. However, HDFS does not provide any scheme to encrypt user's data. Hence, user needs to encrypt his data locally before transferring to HDFS if he requires high privacy. This paper proposes a novel method to encrypt file while being uploaded. Data read from file is transferred to HDFS across a buffer. In our approach, an encryption, which is transparent to user, is applied to the buffer's data before being sent to an out stream to write to HDFS. Our experiment shows that the time needed for the whole process is much less than one of the conventional method. The speedup can be as high as 1.4 in case of 32MB file's size.