The Improved SVM Algorithm Based on MapReduce and Its Implemention on Spam Filtering
Jinfu Chen · Wireless Communication Technology · 2013
Aiming at that the massive text e-mail mining filter requires more storage space and greater computing power,a method of implementing spam filtering based on Hadoop platform is proposed.The data is merged into one big file to be processed;the feature words of every email is selected according to the evaluation function to create the txt vector and convert the e-mail to a structured description.Using the improved SVM based on MapReduce to distribute the load into clusters,and solving the optimal plane using the whole cluster computing power.The experiments show that the improved SVM algorithm can take advantage of the cheap computer cluster to replace expensive high performance machine to implement e-mail mining filter;and the classification efficiency is improved fast with expansion of the cluster scale.