Data Stream Warehousing In Tidalrace.
Theodore J. Johnson, Vladislav Shkapenyuk · Conference on Innovative Data Systems Research · 2015
data is a ubiquitous feature of large modern en terprises. Many organizations generate huge amounts of on-line stre aming data - examples include network monitoring, Twitter feeds, financial data, and industrial application monitoring. Maki ng effective use of these data streams can be challenging. While Da ta Stream Management Systems can provide support for real-time alerting and data reduction, many applications require compl ex analytics on a data history to best make use of the streams. We have been developing technologies for data stream warehousing , starting with the DataDepot (14) system. A data stream warehouse continually ingests data streams, computes complex derived data products, and stores long term histories. To take advantage of new technologies, we have developed a next- generation data stream warehousing system. In this paper we describe the Tidalrace system, our motivations for developing it, and architectural features of Tidalrace that suppor t data stream warehousing.