Comparison Research of Segmentation Performance for Chinese Analyzers Based on Lucene
Chen Qian · Jisuanji gongcheng · 2012
The segmentation performance on Chinese analyzer of Lucene is insufficient,and the third party analyzer is difficult to choose.Because of this problem,this paper introduces several kinds of support Lucene analyzer,based on the experiment,sentence segmentation,word segmentation speed,index space and time,retrieval results and speed of retrieval are compared and researched.Analysis results show that,in Lucene framework,Paoding analyzer based on dictionary segmentation has the best overall performance,one-word analyzer of Lucene has the highest segmentation speed,imdict and ICTCLAS4J analyzer have greater room for improvement on the algorithm efficiency.