OAMS: A Highly Reliable Metadata Service for Big Data Storage
Jiang Zhou, Jing Ming Guo, Weiping Wang, Cuilan Du, Xiaoyan Gu, Dan Meng · 2013
As application requirements increase in quantity and popularity, big data storage is becoming an important technology which data centers and Internet companies depend on. The cluster file system with centralized metadata management often encounters planned or unplanned downtime which requires higher reliability for metadata service. Current paradigms use the backup server to take over as the primary when the latter is in the case of failures. But if the backup crashes, the file system is still in an unreliable state. In this paper, we present a novel primary-backup policy (OAMS), which ensures the availability of metadata service in cluster file system. Different from traditional paradigms, OAMS employs multiple standbys to tolerate the single point of failure. It is based on the built-in shared storage pool for metadata synchronization and a series of protocols for active election, active-standby switching and etc. By using a prepared, automatic state transition among metadata servers, OAMS achieves an automatic recovery in the form of hot standby. It also supports server self-recovery and dynamical addition for standbys at runtime. Evaluation results show that OAMS obviously improves the reliability of metadata service while the average performance degradation is below 8%.