Self-Acting Load Balancing with Parallel Sub File Migration for Parallel File System
Bin Dong, Xiuqiao Li, Limin Xiao, Li Ruan, YU Bin-bin · 2010
With number of I/O servers increasing, the load imbalance among them becomes an emerging challenge in using high-end computing system effectively. But most of contemporary file systems like PVFS2 and Lustre lack load balancing to optimize parallel I/O performance when facing large number of I/O servers. To address this problem, this paper proposes self-acting load balancing for parallel file systems. The proposed self-acting load balancing builds on dynamic file migration, and borrows the ideas from intelligent computing to enable automatic load balancing. Furthermore, in order to support promising scalability, the self-acting load balancing is fundamentally based on distributed architecture, which also permits parallel file migrations to accelerate balancing progress. By taking PVFS2 as basic platform, we implement the self-acting load balancing, and the experiments show that PVFS2 with self-acting load balancing can reduce the average response time and improve performance of parallel I/O obviously compared with original PVFS2.