Prompt Tuning for Text Detoxification
SberDevices Moscow, Russia, Nikita Konodyuk · Computational Linguistics and Intellectual Technologies · 2022
Text detoxification is a challenging style transfer task, that implies paraphrasing into a neutral form while preserving the meaning as closely as possible. In this paper, we present a lightweight approach based on a recently proposed prompt tuning technique. Using RuGPT3-XL (Generative Pretrained Transformer-3 for Russian) as a frozen backbone, we train only a sequence of continuous embeddings inserted before and after an input text. Even though the number of trainable parameters is less than 0.025% of their total number, our approach achieves competitive performance compared to the methods involving full model tuning and ranks 4th on the leaderboard of shared RUSSE Detox task.