A Framework for Fake Review Annotation
Somayeh Shojaee, Azreen Azman, Masrah Azrifah Azmi Murad, Nurfadhlina Mohd Sharef, Nasir Sulaiman · international conference on Modelling and simulation · 2015
The effectiveness of opinion mining relies on the availability of credible opinion for sentiment analysis. Often, there is a need to filter out deceptive opinion from the spammer, therefore several studies are done to detect spam reviews. It is also problematic to test the validity of spam detection techniques due to lack of available annotated dataset. Based on the existing studies, researchers perform two different approaches to overcome the mentioned problem, which are to hire annotators to manually label reviews or to use crowd sourcing websites such as Amazon Mechanical Turk to make artificial dataset. The data collected using the latter method could not be generalized for real world problems. Furthermore, the former method of detecting fake reviews manually is a difficult task and there is a high chance of misclassification. In this paper, we propose a novel technique to annotate review dataset for spam detection by providing more information and meta data about both reviews and reviewers to the annotators for effective spam annotation. We proposed a framework and developed an on-line annotation system to improve the review annotation process. The system is tested for several reviews from the amazon.com and the results is promising with 0.10 error on labeling.