A High-Performance Distributed File System for Mass Data
Shuaiyu Jin, Qing Wen Yan, Yue Zhang, Jie Yang · 2020
With the rapid development of the mobile Internet and cloud computing, storage sub system's pressure increasing fast as the critical process needs more powerful engines. The paper realizes a high-performance distributed file system based on raft protocol. Our file system guarantees the reliability of data through replication. Write-Ahead Logging (WAL) and raft consensus algorithm provide atomicity, durability and file distribution. In order to solve the problem of data fragmentation in distributed systems, we design an automatic fragmentation and load balancing mechanism based on consistent hashing. Each file goes through the raft state machine so that a few machines crash without I/O error. We also implement the FUSE interface for the users to mount and work on files. Performance measurements under some typical workloads show that our file system has excellent I/O performance.