Storage Matters: Evaluating the Impact of Big Data Transfer Techniques on Storage Performance
Adam H. Villa · 2013
When it comes to transferring Big Data, there are two main areas of concern: network performance and storage performance. The primary focus of recent work has been devoted to the problems of network connectivity and bandwidth. Different transfer techniques have been proposed to quickly move massive amounts of data between computers. The goal of these techniques is to maximize bandwidth consumption by any means necessary. The network performance of these techniques has been analyzed; however their impact on storage performance is not thoroughly investigated. In this study, Big Data transfers are evaluated from the storage viewpoint. Particular attention is focused on the granularity of request sizes issued to a storage node. This paper illustrates that there is a significant impact on performance when small portions of a data set are requested in place of a single large request.