Robust detection of comment spam using entropy rate
Alex Kantchelian, Justin H. Ma, Ling Huang, Sadia Afroz, Anthony Douglas Joseph, J. D. Tygar · 2012
In this work, we design a method for blog comment spam detection using the assumption that spam is any kind of uninformative content. To measure the "informativeness" of a set of blog comments, we construct a language and tokenization independent metric which we call content complexity, providing a normalized answer to the informal question "how much information does this text contain?" We leverage this metric to create a small set of features well-adjusted to comment spam detection by computing the content complexity over groupings of messages sharing the same author, the same sender IP, the same included links, etc.