Performance analysis of a fault-tolerant exact motif mining algorithm on the cloud
Nhan Nguyen, Mohammad Maifi Hasan Khan · 2013
In this paper, we present the performance analysis and design challenges of implementing a fault-tolerant parallel exact motif mining algorithm leveraging the services provided by the underlying cloud storage platform (e.g., data replication, node failure detection). More specifically, first, we present the design of the intermediate data structures and data models that are needed for effective parallelization of the motif mining algorithm on the cloud. Second, we present the design and implementation of a fault-tolerant parallel motif mining algorithm that enables the data analytic system to recover from arbitrary node failures in the cloud environment by detecting node failures and redistributing remaining computational tasks in real-time. We also present a data caching scheme to improve the system performance even further. We evaluated the impact of various factors such as the replication factor and random node failures on the performance of our system using two different datasets, namely, an EOG dataset and an image dataset. In both cases, our algorithm exhibits superior performance over the existing algorithms, thus demonstrating the effectiveness of our presented system.