A Universal Namespace Approach to Support Metadata Management and Efficient Data Convergence of HPC and Cloud Scientific Workflows
Hsing-bung Chen · 2018
In this paper, we present an innovative namespace architecture called the Universal Namespace (UNS). UNS enhances both the local and global namespace with portable, mobile, and exchangeable properties and supports fetch-from-anywhere storage systems without requiring remote mounting file systems. UNS exploits the extended file attributes of file systems and enhances file system metadata with unified remote data access information and methods. We design the proposed UNS to support big data computing on multiple geolocation-based shared storage systems. We describe the innovative concepts of the proposed UNS's architecture and demonstrate UNS's performance results from various test cases. Finally, we summarize the benefits and advantages of using UNS in supporting big data convergence of HPC and Cloud scientific workflows. Furthermore, we present the future design and developmental direction of UNS.