Parallelization of FP-growth Algorithm for Mining Probabilistic Numerical Data Based on MapReduce
Bin Pei, Xiuzhen Wang, Wang Fenmei · 2016
Many association rule mining algorithms find associations and correlations from traditional transaction databases, in which the content of each transaction is definitely precise. However, due to instrument errors, imprecise of sensor monitoring systems, and so on, real-world data tend to be numerical data with inherent uncertainty. To deal with these situations, we propose a FP-growth-based mining algorithm PNFP-growth to efficiently find association rules from probabilistic numerical data, where each numerical item in the transactions is associated with an existential probability. In addition, to deal with big data situation, we also introduce a parallelized PNFP-Growth in the MapReduce framework, which scales well with the size of the dataset while minimizing data replication and communication cost.