Tuning the performance of I/O-intensive parallel applications
Anurag Acharya, Mustafa Uysal, Robert L. Bennett, Assaf Mendelson, Michael D. Beynon, Jeff Hollingsworth, Joel Saltz, Alan Sussman · 1996
Getting good I/0 performance from parallel programs is a critical problem for many application domains.In this paper, we report our experience tuning the I/0 performance of four application programs from the areas of satellite-data processing and linear algebra.After tuning, three of the four applications achieve application-level 1/0 rates of over 100 MB/s on 16 processors.The total volume of 1/0 required by the programs ranged from about 75 MB to over 200 GB.We report the lessons learned in achieving high I/0 performance from these applications, including the need for code restructuring, local disks on every node and knowledge of future I/0 requests.We also report our experience on achieving high performance on peer-to-peer configurations.Finally, we comment on the necessity of complex I/0 interfaces like collective I/0 and strided requests to achieve high performance.