Coordinating Parallel Hierarchical Storage Management in Object-base Cluster File Systems
Dingshan He, Xianbo Zhang, David Du, Gary Grider · 2011
Object-based storage technology enables building large-scale and highly-scalable cluster file systems using commodity hardware and software components. On the other hand, a hierarchy of storage subsystems with different costs and performance should be incorporated into such systems to make them affordable and cost-effective. Existing SAN-based (block-based) cluster solutions suffer from slow data movement between storage levels due to their single-archiving-point architecture. In this paper, we propose a novel parallel data moving architecture in objectbased cluster file systems. Data movements are coordinated and performed in parallel between multiple pairs of storage subsystems. In addition, data movements are fully automated and transparent to users. Our proposed parallel data moving architecture is prototyped on the Lustre file system. Performance study shows that our scheme can scale up easily by adding more pairs of hierarchical storage devices. 1.