A Parallel MPI I/O Solution Supported by Byte-addressable Non-volatile RAM Distributed Cache
Artur Malinowski, Paweł Czarnul, Piotr Dorożyński, Krzysztof Czuryƚo, Łukasz Dorau, Maciej Maciejewski, Paweł Skowron · Annals of Computer Science and Information Systems · 2016
While many scientific, large-scale applications are data-intensive, fast and efficient I/O operations have become of key importance for HPC environments.We propose an MPI I/O extension based on in-system distributed cache with data located in Non-volatile Random Access Memory (NVRAM) available in each cluster node.The presented architecture makes effective use of NVRAM properties such as persistence and byte-level access behind the MPI I/O API.Another advantage of the proposed solution is making development of a parallel application easy and efficient as a programmer just needs to use the well known MPI I/O data model and API while efficient file access is automatically provided without a need for application level optimizations like avoiding frequent operations on a small data.Results of experiments obtained with three different applications suggest, that the extension significantly reduces file access time, especially for small I/O operations.By locating cache facilities on computing nodes, the extension decreases load of file system servers and makes I/O scalable.