Text Segmentation Based on Word Embedding on Indonesian Quran Translation by Greedy with Window Approaching
Ramdani Ramdani, Arief Fatchul Huda, Moch Arif Bijaksana · 2019
This paper proposes an approach to improve segmentation based on word embedding. The Greedy splitting is a major of the element in text segmentation. The previous weakness of this method, we need to define how many segments should be provided(K). The nature of segmentation does not provide the number of segments (K). In this research, greedy splitting was improved by applying window approaching, to minimize the greedy process by defining the number of sentences or words that would be examined. We assumed there only two segments on the local block or one splitting by getting the lowest score. We conducted the experiments by Quran as dataset and compared with other segmentation algorithms. The results showed that the proposed method was average at 0.68 (WindofDiff). Whereas TextTiling was reached 0.74, C99 at 0.743 and Original Greedy at 0.83 also the number of K segments could be searched by itself. Moreover, we found phenomena in the uselessness of stopword removal had an effect was indicated by decreased WindowDiff.