Building a Mass Storage System for Physics Applications
Harvard H. Holmes, Stewart C. Loken · University of North Texas Digital Library (University of North Texas) · 1991
The IEEE Mass Storage Reference Model and forthcoming standards based on it provide a standardized architecture to facilitate designing and building mass storage systems, and standard interfaces so that hardware and software from different vendors can interoperate in providing mass storage capabilities. A key concept of this architecture is the separation of control and data flows. This separation allows a smaller machine to provide control functions, while the data can flow directly between high-performance channels. Another key concept is the layering of the file system and the storage functions. This layering allows the designers of the mass storage system to focus on storage functions, which can support a variety of file systems, such as the Network File System, the Andrew File System, and others. The mass storage system provides location-independent file naming, essential if files are to be migrated to different storage devices without requiring changes in application programs. Physics data analysis applications are particularly challenging for mass storage systems because they stream vast amounts of data through analysis applications. Special mechanisms are required, to handle the high data rates and to avoid upsetting the caching mechanisms commonly used for smaller, repetitive-use files. High data rates are facilitated by direct channel connections, where, for example, a dual-ported drive will be positioned by the mass storage controller on one channel, then the data will flow on a second channel directly into the user machine, or directly to a high capacity network, greatly reducing the I/O capacity required in the mass storage control computer. Intelligent storage allocation can be used to bypass the cache devices entirely when large files are being moved.