Anthill: A Scalable Run-Time Environment for Data Mining Applications
Renato Ferreira, Wagner Meira, Dorgival O. Guedes, Lúcia M. A. Drummond, Bruno Cardoso Coutinho, George Teodoro, Tulio Tavares, Raquel Linhares Bello de Araújo, G.T. Ferreira · 2006
Data mining techniques are becoming increasingly more popular as a reasonable means to collect summaries from the rapidly growing datasets in many areas. However, as the size of the raw data increases, parallel data mining algorithms are becoming a necessity. In this paper, we present a run-time support system that was designed to allow the efficient implementation of data-mining algorithms on heterogeneous distributed environments. We believe that the runtime framework is suitable for a broader class of applications, beyond data mining. We also present a parallelization strategy that is supported by the run-time system. We show scalability results of three different data-mining algorithms that were parallelized using our approach and our run-time support. All applications scale almost linearly up to a large number of nodes.