A modified hybrid Fuzzy clustering method for big data
Amir Khoshkbarchi, Ali Reza Kamali, Mehdi Amjadi, Maryam Amir Haeri · 2016
Clustering is among the most common data mining techniques and Fuzzy clustering can model the world even more realistically and more precisely. One of the most favorable fuzzy clustering methods is the Fuzzy C-Means (FCM) algorithm, which is actually identical to the (original) K-Means clustering algorithm fueled with a fuzzy flavor. However, there are some issues with the fuzzy clustering methods; FCM is too sensitive to the initial conditions, like identifying the initial arrangement of clusters. Furthermore, it suffers from a slow convergence and the lack of a guarantee to reach a global optimal solution and it faces problem in the handling big data. Dealing with these problems, and for the sake of an improved precision, this paper proposes a new modified FCM algorithm, based on Particle Swarm Optimization (PSO) with considering multiple dimensions of the data. Besides that, the proposed method utilizes the Map-Reduce technique in order to make the new method suitable for big data. This objective is achieved by selecting the most optimal candidates in the mapping phase and aggregating them in the reduce phase with the purpose of getting the most optimal results. Simulation and experimental results show an improved performance for the proposed method applied to big data, compared to other hybrid FCM-PSO method.