ClearView: Data cleaning for online review mining
Amanda Minnich, Noor Abu-el-rub, Maya B. Gokhale, Ronald G. Minnich, Abdullah Mueen · 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · 2016
How can we automatically clean and curate online reviews to better mine them for knowledge discovery? Typical online reviews are full of noise and abnormalities, hindering semantic analysis and leading to a poor customer experience. Abnormalities include non-standard characters, unstructured punctuation, different/multiple languages, and misspelled words. Worse still, people will leave “junk” text, which is either completely nonsensical, spam, or fraudulent. In this paper, we describe three types of noisy and abnormal reviews, discuss methods to detect and filter them, and, finally, show the effectiveness of our cleaning process by improving the overall distributional characteristics of review datasets.