Application of a mass storage system to science data management
O. Graf, M. Jones, F. Sisco · 2002
Approaches to the issues of data ingestion, data restructuring, physical data models, relationship file structure to data system performance, data product generation, data transfer to remote users, data subset extraction, data browsing, and user interface have been examined. The High Performance Data System architecture provides an environment for bringing together the technologies of mass storage, large bandwidth data networks, high-performance data processing, and intelligent data access. The prototype system demonstrates an approach to these issues. In addition, the design process has defined some important requirements for the mass storage file system, such as logical grouping of files, aggregate file writes, and multiple dynamic storage device hierarchies.>