Distributed large data-object management architecture
William E. Johnston, Jin Guojun, Jason Lee, Mary Ruth Thompson, Brian Tierney · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · 1997
We are exploring the use of highly distributed computing and storage architectures to provide all aspects of collecting, storing, analyzing, and accessing large data-objects. These data-objects can be anywhere from tens of MBytes to tens of GBytes in size. Examples are: single large images from electron microscopes, video images such as cardio- angiography, sets of related images such as MRI data and images and numerical data such as the output from a particle accelerator-experiment. The sources of such data objects are often remote from the users of the data and from available large-scale storage and computation systems. Our Large Data- object Management system provides network interface between the object sources, the data management system and the user of the data. As the data is being stored, a cataloguing system automatically creates and stores condensed versions of the data, textual metadata and pointers to the original data. The catalogue system provides a Web based graphical interface to the data. The user is able the view the low- resolution data with a standard internet connection and Web browser, or if high-resolution is required can use a high- speed connection and special application programs to view the high-resolution original data.