Web spam detection based on SMOTE and random forests
Shuang Gao · Journal of Shandong University · 2013
Web spam refers to the actions intended to mislead search engines into ranking some pages higher than they deserved,which could significantly deteriorate the quality of searching results.Considering the serious imbalance of the Web spam dataset,it was proposed to use over-sampling method SMOTE to balance the dataset,then to train the classifiers with random forests algorithm.The results showed that the SMOTE+RF method was more prominent by means of experimental comparison with the conventional single classifiers and the ensemble learning classifiers.The important parameters of this method were optimized based on experimental results,and the reasons for the improvement of the AUC value after using SMOTE were also analyzed.Experimental results on WEBSPAM UK2007 dataset showed that this method could markedly improve the performance of the classifiers,of which the AUC value could exceed the best result of Web Spam Challenge 2008.