Taming parallel I/O complexity with auto-tuning
Babak Behzad, Huong Vu Thanh Luu, Joseph Huchette, Suren Byna, Prabhat, Ruth A. Aydt, Quincey Koziol, Marc Snir · 2013
We present an auto-tuning system for optimizing I/O performance of HDF5 applications and demonstrate its value across platforms, applications, and at scale. The system uses a genetic algorithm to search a large space of tunable parameters and to identify effective settings at all layers of the parallel I/O stack. The parameter settings are applied transparently by the auto-tuning system via dynamically intercepted HDF5 calls.