Optimizing collective I/O performance on parallel computers
Ying Chen, Jarek Nieplocha, Ian T Foster, Marianne Winslett · 1997
While individual parallel I/O systems can incorporate sophisticated techniques and achieve impressive performance in particular situations, researchers as yet have only limited m~derstanding of the impact of various design decisions or of the techniques required for performance robustness.One remedy is to perform detailed comparative studies of different I/O libraries.In this paper, we describe such a study for the Disk Resident Array and Panda libraries, both designed to support high-performance I/O for arrays.While the two systems have many similarities, their designs and implementations are based on different assumptions and target different applications.We base our study on two I/O structures commonly encountered in scientific applications: the collective read/write of an entire array and the collective read/write of an arbitrary array section.Experiments are performed on two parallel file systems (IBM PIOFS and Intel PFS) antI one commodity Unix file system (AIX JFS).Our results yield insights into