An Unsupervised Method for Coordinated Sentence Boundary and Proper Noun Detection in Turkish
Can Özbey, Onur Sahil Cerit · 2022 30th Signal Processing and Communications Applications Conference (SIU) · 2022
In this paper, an unsupervised approach is proposed for disambiguating sentence boundaries in natural language text imposed by the period character in coordination with proper noun status of the subsequent word. In this respect, a local hidden Markov model having latent states as to sentence boundary and noun categories is trained where maximum likelihood estimation of model parameters are obtained based solely on the orthographic rules of capitalization. Its performance was evaluated in comparison with several sentence segmentation approaches in the literature yielding almost equally good results as one of the state-of-the-art transformer-based deep learning models.