Improvement of the Dotplotting Method for Linear Text Segmentation
Na Ye, Jingbo Zhu, Haitao Luo, Huizhen Wang, Bin Zhang · 2006
The dotplotting method, employed by Reynar (1994), is a state-of-the-art algorithm for automatic linear text segmentation. However, several problems are found in its measure for assessing density that represents topical coherence: the density function is asymmetric, leading to the apparent false conclusion that forward scan may result in different segmentation with backward scan; besides, while determining next boundary, the assessing strategy doesn't adequately take the previously located boundaries into account. In this paper we propose modified models that remedy these problems. We also make use of segment length to improve segmentation performance. Experimental results show that the modified models achieve considerable improvement in P/sub k/ value and precision and recall over the original dotplotting method.