New Words Discovery in Microblog Content
Huo Shua · 2014
New words discovery is of great significance in the field of natural language processing.It is more difficult to find new words in microblog than in other corpus.In this paper,an algorithm based on context entropy is proposed,and the new word candidates are filtered based on the context.To improve the precision,lexical features are introduced and an algorithm combining them with term frequency is put forward.Thus,the precision rate and the recall rate are greatly improved,and the F-measure value is up to 89.6%.