Big Data Management Performance Evaluation in Hadoop Ecosystem
Qing Liu, Yinjin Fu, Guiqiang Ni, Jianmin Mei · 2017
With the further research of big data management, plenty of components for big data management have been developed. Based on Hadoop platform, these components provide solutions for big data management from different levels. The Hadoop ecosystem has gradually taken its shape. However, users usually lack the knowledge about the features of these components, such as the I/O pattern, capability, application scenes and so on. When dealing with some big data problems, these components are often chosen by user's experience and this will definitely lead to mismatch between the demands and the management tools. Thus, the platform cannot play out its optimal performance. Focus on this issue, this paper tested and evaluated several widely used mainstream big data management tools in Hadoop ecosystem from three levels: distributed file system, NoSQL database and SQL-like component. After the brief introduction to the typical management tools, comprehensive comparisons of these tools of the same level are carried out. The advantages and disadvantages are discussed and their performance are also tested and analyzed.