Baseline evaluation
Farag Saad · 2014
At present, the sentiment analysis task is a relatively recent research field which has led to many inconsistent findings in literature. The debate is two-fold: what is the best performing baseline classifier and what is the most useful feature weighting method e.g., term presence (TP), term frequency (TF), TF-IDF etc., which can be used to improve a classifier's performance. Naïve Bayes, with its variations and Support Vector Machine are a commonly used baseline in the sentiment analysis task. However, their reported performance varies among researchers and has led to a divergence as to which is the best baseline classifier that can be used in the sentiment analysis task. In order to shed some light on this controversy, we have conducted a series of widely comparative experiments (including twelve various domains) to evaluate the performance of various machine learning classifiers (Naïve Bayes with its variations, Support Vector Machine and J48 - an implementation of decision tree-based learning -) in the sentiment analysis task. The experimental results indicate that the Binarized Multinomial Naïve Bayes (BMNB) classifier exhibits the best performance in a short snippet sentiment analysis task. Furthermore, the classification performance, using feature selection methods, namely the information gain method (IG), has been significantly improved.