Probabilistic anti-spam filtering with dimensionality reduction
Tiago A. Almeida, Akebo Yamakami, Jurandy Almeida · 2010
One of the biggest problems of e-mail communication is the massive spam message delivery. Everyday, billion of unwanted messages are sent by spammers and this number does not stop growing. Helpfully, there are different approaches able to automatically detect and remove most of these messages, and a well-known ones are based on Bayesian decision theory. However, many machine learning techniques applied to text categorization have the same difficulty: the high dimensionality of the feature space. Many term selection methods have been proposed in the literature. Nevertheless, it is still unclear how the performance of naive Bayes anti-spam filters depends on the methods applied for reducing the dimensionality of the feature space. In this paper, we compare the performance of most popular methods used as term selection techniques with some variations of the original naive Bayes anti-spam filter.