Sensational Headline Identification By Normalized Cross Entropy-Based Metric
Zhen Yang, K. Gao, K. Fan, Ying‐Cheng Lai · The Computer Journal · 2014
Nowadays multimedia social networks are fueled by sensational coverage of sex, violence and crime. In this paper, we provide a normalized cross entropy metric to determine whether a headline is a sensational headline or not by the literal consistency between the headline and its corresponding document. Experiments on a Chinese data set show that the traditional relevancy measurements—vector cosine, relative entropy, likelihood and cross entropy—suffer from strong dependence on text length and are unable to effectively identify sensational headline. The experimental results on both Chinese data sets and English data sets show that our metric can cover the positive effects of high-frequency words and overcome the negative effects of the lengths of the title and the document.