University of Regensburg @ SwissText 2021 SEPP-NLG: Adding Sentence Structure to Unpunctuated Text
Gregor Donabauer, Udo Kruschwitz · University of Regensburg Publication Server (University of Regensburg) · 2021
This paper describes our approach (UR- mSBD) to address the shared task on Sentence End and Punctuation Prediction in NLG Text (SEPP-NLG) organised as part of SwissText 2021. We participated in Subtask 1 (fully un- punctuated sentences – full stop detection) and submitted a run for every featured language (English, German, French, Italian). Our sub- missions are based on pre-trained BERT mod- els that have been fine-tuned to the task at hand. We had recently demonstrated, that such an ap- proach achieves state-of-the-art performance when identifying end-of-sentence markers on automatically transcribed texts. The difference to that work is that here we use language- specific BERT models for each featured lan- guage. By framing the problem as a binary tagging task using the outlined architecture we are able to achieve competitive results on the official test set across all languages, with Re- call, Precision, F1 ranging between 0.91 and 0.96 which makes us joint winners for Recall in two of the languages. The official baselines are beaten by large margins.