Predicting energy consumption with StreamMine3G
André Martin, Rodolfo Marinho, Andrey Brito, Christof W. Fetzer · 2014
In this paper, we present our approach on solving the DEBS Grand Challenge using StreamMine3G, a distributed, highly scalable, elastic and fault tolerant ESP system. We will provide an overview about the system architecture of Stream-Mine3G and implementation details of an application aimed at consumption prediction and outlier detection. Using our elastic approach, we can provide an accurate prediction as we can keep a practically unbounded history able to deal with high volume, highly fluctuating workloads. Our system also provides techniques for dealing with incomplete data in the source stream, which is a common problem when processing data from a large number of sources. Finally, we provide performance measurements showing that we are able to process the dataset given as part of the 2014 DEBS Challenge (135 GB) at a throughput of up to 40 kEvents/s.