Grid-Based Zero-Latency Data Warehousing for Continuous Data Streams Processing.
Tho Manh Nguyen, A Min Tjoa · Information Integration and Web-based Applications & Services · 2004
Continuous data streams are information sources in which data arrives in high-volume, in unpredictable rapid bursts. Processing data streams, due to the lack of resource, is challenging in that (1) random access to fast and large data streams may be impossible, (2) the exact answers from data streams are often too expensive. The Grid Technology and Zero-Latency Data Warehousing concepts are applied in the framework of building a Grid-based Zero-Latency Data Stream Warehouse (GZLDSWH) to overcomes the resource limitation issue in data stream processing without using traditional approximate approaches. The GZLDSWH is built upon a set of Open Grid Service Infrastructure (OGSI)-based grid services and Globus Toolkit 3 (GT3) with the capable of storing continuous data streams, making analytical processing, and reacting autonomously in near real time with some kinds of events based on well-established Knowledge Base. The architecture of a GZLDWH, the operations and the collaboration between the grid services and several challenging issues are the main topics addressed in the paper.