High-performance GRID Database Manager for Scientific Data
Tore Risch, Milena Gateva Koparanova, B. Thidé · 2002
The GRID initiative provides an infrastructure for distributed computations among widely distributed high-performance computers. This will allow for exchanging and processing very large amounts of data. The LOFAR project (www.nfra.nl/lofar) is an international initiative to build a versatile, geographically distributed, multi-point radio facility for astrophysics, space physics, atmospheric physics, and radio research, utilizing very high performance GRID computing. LOIS is a proposed Swedish outrigger to LOFAR providing a software radar. As the volume of processed data by LOFAR/LOIS is very large and dynamic there will be need for very high performing data management systems. For this a high-performance stream-oriented distributed data manager and query processor is being developed that allows very efficient execution of database queries to streamed data involving numerical and other data. Very high performance is attained by utilizing many object-relational main-memory database engines running on PCs and connected through the GRID. The project leverages upon a highperformance, extensible, and object-oriented database engine, the Amos II kernel, developed in the Uppsala Database Laboratory. A very high performing stream-oriented DBMS is being developed for representing and querying non-relational data representations extracted from the data flows used in space and environmental physics applications. Of particular interest is the development of new distributed data population and query processing techniques for this kind of applications and thereby utilizing distributed and scalable data structures for high-performance stream data processing.