A Dynamic Metadata Equipotent Subtree Partition Policy for Mass Storage System
Gongye Zhou, Qiuju Lan, Jincai Chen · 2007
Metadata plays an important role in mass storage system. How to distribute and balance the metadata of the metadata server cluster determines the overall performance of a cluster. Two popular metadata distribution policies are the dynamic subtree policy and hashing policy, while the dynamic subtree partition is vulnerable to the imbalance workload and hashing partition has a random distribution which will incur a burst of network overhead when updating metadata. We present a novel approach for metadata management. It combines hash and subtree partitioning policies together to partition directory hierarchy tree into equipotent subtrees with a certain granularity and employs value of hashing subtree to distribute subtrees across the metadata servers. It also employs a balance strategy to adjust the metadata distribution dynamically. After adjustment, we present a hot spots elimination strategy to detect and reclaim hot spots in the file system efficiently. We also demonstrate a design using this strategy to achieve more efficient performance than the other policies using hashing partitioning and subtree partitioning purely.