Integrating High Performance File Systems in a Cloud Computing Environment
Abhisek Pan, John Paul Walters, Vijay S. Pai, Dong In D. Kang, Stephen P. Crago · 2012
This paper outlines our ongoing efforts to effectively integrate a parallel file system in a cloud environment. We investigate how a parallel file system (PFS) can be effectively integrated and provided as a service to cloud users running High Performance Computing (HPC) applications, and what would be the performance and security implications of such a service. A critical requirement of running today's dataintensive HPC applications is the availability of a parallel high performance storage system. The objectives of our work are two-fold: first, to identify and evaluate the possible ways to deploy a PFS in a cloud infrastructure, and second, to design a framework to provide the PFS as a service inside a cloud-management framework which would allow users to provision and configure PFS storage dynamically and securely, in much the same way they can provision instances and volumes. Initial results indicate that among the options evaluated, the best filesystem performance can be obtained by providing a native parallel file-system installation as a service and using file-system passthrough from the virtual machine instances to access the files directly.