High performance access to radio astronomy data: a case study
John F. Karpovich, James C. French, Andrew S. Grimshaw · 2002
As CPU performance has rapidly improved, increased pressure has been placed on the performance of accessing external data in order to keep up with demand. Increasingly often the I/O subsystem and related software is unable to meet this demand and valuable CPU resources are left underutilized, while users are forced to wait longer than necessary for results. This problem is especially acute for many scientific applications which use large data sets and require high performance. The paper presents experiences with working to alleviate an I/O bottleneck in radio astronomy applications at the National Radio Astronomy Observatory (NRAO). It presents the model, the ExtensibLe File Systems model (ELFS), for improving both the performance of data access and the usability of access software. It then presents the current situation at NRAO, the solution, performance results and plans for future work.>