Sentence Boundary Detection in Turkish News with Regular Expressions
Can Özbey, Özge Dinҫsoy · 2019
In this study, a rule-based sentence boundary detection method for Turkish news is developed. Incorporating direct quotations as well, which have not been previously addressed within the scope of the problem, disambiguation of punctuation marks with respect to sentence boundaries is carried out by means of a single regular expression. The performance of the proposed method is compared with the statistics-based Punkt algorithm in terms of prediction speed and accuracy, applied on news articles collected from the Web. Also, the generated regular expression and annotated test set are made publicly available as a baseline for further research.