A Comparative Study of MPI-IO Libraries for Offloading of Collective I/O Tasks
Ayşe Bağbaba · 2021 International Conference on Engineering and Emerging Technologies (ICEET) · 2021
Many scientific applications running on supercomputers deal with increasingly large data sets and are correspondingly bottlenecked by the time it takes to perform input and output (I/O) of data from/to the file system. In principle, user's may rearrange file access patterns and data layout, both, in the application and on the file system to increase I/O performance. Message Passing Interface I/O (MPI-I/O) acts as a middle-ware between application and file system. It provides derived data types and file views to describe different access patterns as well as I/O hints in form of tuning parameters for different file systems. This study compares two MPI-I/O libraries; ROMIO and OMPIO based on the application's access pattern and the underlying file system. Special focus is put on collective I/O and it's potential performance improvements over individual I/O operations. The study shows that one cannot reliably choose a single data layout and expect uniform performance portability among these two libraries. The results can provide insights into the metrics that impact I/O performance significantly and help developers to investigate I/O behaviour of HPC applications.