Detoxification of Russian texts based on combination of controlled generation using pretrained ruGPT3 and the Delete method
Ekaterina V. Totmina · Computational Linguistics and Intellectual Technologies · 2022
This article describes our solution for the RUSSE Detoxification 2022 text automatic detoxification competition held as part of the Dialogue 2022 conference. Our approach consisted in filtering the p rovided training data set, fine-tuning the pretrained ruGPT3 model and selecting examples of detoxified (neutral) sentences generated with its help based on their cosine proximity and ROUGE-L to the input toxic sentence for their subsequent processing using the ruPrompts library for ruGPT-3. The final stage of processing the generated neutral comments was carried out using the Delete method - an uncontrolled detoxification model based on rules, which deleted all the remaining coarse and absentee words stored in the dictionary provided by the organizers. At the Human Evaluation stage, the system received a chrF metric value of 0.455; at the Automatic Evaluation stage - 0.505, and took eighth place at Manual Evaluation. We conducted a review and analysis of examples of detoxified sentences obtained using our model. The analysis showed that some of the generated neutral sentences in most cases lose the meaning of the original toxic sentence, and also retain either a full negative connotation or a partial one.