Data Placement Algorithm for Large-scale Storage System
Shen Zheng · 2013
With the era of big data coming,t PB and EB even ZB-level dataset makes storage system scalable.Traditional data distribution algorithm was confronted with serious challenge because of different performance storage devices added and the old ones quitted,even multiple devices failed simultaneously.A new hash mapping algorithm was proposed which supports the node weight and multi-replica and also considers node failure and node overload.The algorithm can adapt dynamically to change of storage nodes and promises data even distribution probabilistically for different performance nodes.Besides,the one can effectively deal with node failure and node overload which can improve the availability and performance of the system.