Distributed algorithm for sequential pattern mining on a large sequence dataset
Tho Hoang, Bac Hoai Le, Minh-Thai Tran · 2017
Sequential pattern mining is an active research area because of its diverse applications. There have been many studies suggesting efficient mining algorithms. However, when the size of the dataset is large, these algorithms have limitations, such as performance and scalability. Therefore, parallel mining on a cluster of computers is a necessary issue to study for the sequential pattern mining problem, as the size of the datasets is increasing. In this paper, we propose a parallel algorithm to deal this problem, called DSPDBV, using dynamic vector bit structures on the MapReduce distributed programming model. In addition, the algorithm uses different techniques to prune redundant candidates early and reduce the amount of memory usage. Experimental results show that DSPDBV is highly efficient and scalable for large sequence datasets.