File System Monitoring as a Window Into User I/O Requirements
Andrew Uselton, Katie Antypas, Daniela Mayumi Ushizima · 2010
As part of the 2008-2009 upgrade of the Franklin Cray XT at the National Energy Research Scientific Computing center, its I/O resources were more than doubled. The extra bandwidth reduced contention for I/O resources and those resources were split into two file systems, /scratch and /scratch2, each with more bandwidth than the original /scratch. In an effort to improve responsiveness, NERSC consulting encouraged ”big I/O ” jobs to use /scratch2. File system monitoring data indicate that this division of the NERSC workload has been largely successful, and we present several useful characterizations of the two I/O workloads that resulted. This feedback for both the users and the center enhances NERSC’s ability to manage and provision Franklin’s I/O subsytem as well as to plan for future I/O requirements. The workload characterization presented here also lays the groundwork for the development of a rigorous methodology for measuring the impact of I/O on system utilization, which we outline in Future Work.