The Recognition Method of Unknown Chinese Words Based On Fragments Segmentation
Lei Zhou · Journal of Changshu Institute of Technology · 2007
The paper makes a research on unknown Chinese word recognition and introduces a new method to extract unknown Chinese words from texts.Firstly it segments the full text and combines the adjacent single Chinese character to short strings(fragments),and then it uses full-segmentation method to divide each fragment into strings,and each string is assigned a term weighting by rules and frequency.Lastly it uses the greedy algorithm to get the longest path of each fragment so as to get the unknown word from every string in this path.The precision on the open test sets is 82.88% and recall is 87.51%.