Application of Chinese Word Segmentation Based on Linguistic Environment Analysis in Text Information Filtering System
Zhi-an Yi, Jia Lv · 2009
This paper provides Chinese word segmentation based on language analysis problem in text information filtering system. The improved Chinese word segmentation is made of a bigram segmentation and a segmentation correction, new words recognition and disambiguation through the bigram segmentation, check the accuracy of segmentation results using the segmentation correction from the perspective of syntax. It has been proved by experiments that the segmentation not only strengthen the system's language analysis ability, but also improve the accuracy of text information filtering system when the improved Chinese word segmentation was applied to the text analysis module.