Adaptive IO System (ADIOS)
Jin Chen, Scott Klasky, Stephen W Hodson, Weikuan Yu, Jay Lofstead, Hasan Abbasi, Karsten Schwan, Wei‐keng Liao, Manish Parashar, Ciprian Docan, Ron Oldfield · 2008
ABSTRACT:ADIOS is a state of the art componentization of the IO system that has demonstrated impressive IO performance results on the Cray XT system at ORNL. ADIOS separates the selection and implementation of any particular IO routines from the scientific code offering unprecedented flexibility in the choices for processing and storing data. The API was modelled on F90 IO routines emphasizing simplicity and clarity using external metadata for richness. The metadata is described in a stand-along XML file that is parsed once on code startup and determines what IO routines and parameters are used by the client code for each grouping of data elements generated by the code. By employing this API, a simple change to an entry in the XML file changes the codes to use either synchronous MPI-IO, collective MPI-IO, parallel HDF5, pnetcdf, NULL (no output), or asynchronous transports such as the Rutgers DART implementation and the Georgia Tech DataTap method. Simply by restarting the code, the new IO routines selected in the XML file will be employed. Furthermore, we have been defining additional metadata tags to support in-situ visualization solely through changes in the XML metadata file. The power of this technique is demonstrated on the GTC, GTC_S, XGC1, S3D, Chimera, and Flash codes. We show that when these codes run on a large number of processors, they can sustain high I/O bandwidth when they write out their restart and analysis files. KEYWORDS: I/O, componentization 1.