P-WLPA algorithm research on parallel framework Spark
Shuang Wang, Peng Wu, Tao Liu, Wei Min Kong · 2014
Weighted Label Propagation Algorithm with Probability Threshold (P-WLPA) with typical serial execution prototype is proposed to be applied in data classification. Meanwhile on distributed computing system Spark, parallel P-WLPA algorithm for labeling big data is conducted. The algorithm sets prior conditions when configuring undirected graph and optimizes parameter learning in P-WLPA process. Through experiments, we analyze the relationship between Iterations for convergence, sample stability threshold and error rate. Finally Serial and parallel P-WLPA performance comparision demonstrates the feasibility and efficiency of the parallel P-WLPA algorithm implementation on Spark.