Optimization of operator partitions in stream data warehouse
Marcin Gorawski, Aleksander Chrószcz · 2011
Memory and time optimization is a key task of Stream Data Warehouses (SDWs). StrETL processes in those systems are similar to queries in Data Stream Management Systems (DSMSs). This fact allows us to migrate some methods from DSMS to SDW. We have observed that schedulers and algorithms introduced to create operator partitions are analyzed separately either in StrETL processes or in stream queries. The fact is, those two mechanisms affect each other and it is justified to study potential benefits of combining them together. In the paper we introduce a solution which cooperates with a scheduler in order to create more efficient operator partitions. Another noteworthy issue is that this algorithm is able to optimize a wider range of operator topologies. Finally, experimental evaluation show that our solution allows achieving a smaller memory consumption or a shorter response time in comparison with the competing strategies.