CFS: Enhancing Metadata Management in Cross-Domain File Systems with CRDTs
Qiwen Ke, Yiming Zhang · Proceedings/Proceedings of the ... International Conference on Software Engineering and Knowledge Engineering · 2024
Traditional local disk file systems like Ext4, XFS, and Btrfs excel in managing file data but struggle with metadata-intensive workloads and small file accesses, especially across distributed and cross-domain environments.In this paper, we present CrossFS (CFS), a new cross-domain file system designed to overcome these challenges.CrossFS leverages Conflict-free Replicated Data Types (CRDTs) for efficient metadata synchronization and merging, enhancing both performance and consistency across systems.We organize metadata into a single sparse table using a Log-Structured Merge Tree (LSMT), minimizing metadata overhead and optimizing access efficiency.Additionally, by stacking another local file system as an object store, our system ensures efficient large file allocation and access, while preserving local performance through FUSE integration and rest rpc framework, for cross-regional capabilities.Experimental evaluations demonstrate significant performance enhancements over existing solutions, with our system achieving up to 2× the performance of traditional local file systems and a 40% improvement over distributed systems.