Exploration and Improvement in Keyword Extraction for News Based on TFIDF
Yan Yang, Liang He, Meng Qiu · Energy Procedia · 2011
This paper presents a new keyword extraction algorithm for Chinese news web pages which is bases on the traditional TFIDF method and improves it by combining channel division within and putting word co-occurrence feature into consideration. Word co-occurrence distribution is an important statistical model widely used in natural language processing that reflects the co-relationship of the words. And the channel division can improve the accuracy of IDF value so as to improve the efficiency of TFIDF algorithm. Experiments on randomly selected web pages have been performed to demonstrate the quality of the keywords extracted by our proposed algorithm.