Using RuGPT3-XL Model for RuNormAS competition
Anton Emelyanov, Oleh Shliazhko, Nadezhda Katricheva, Tatiana Shavrina, SberDevices, Sberbank, Moscow, Russia · Computational Linguistics and Intellectual Technologies · 2021
The paper presents a fine-tuning methodology of the RuGPT3-XL (Generative Pretrained Transformer-3 for Russian) language model for the normalization of text spans task.The solution is presented in a competition for two tasks: Normalization of Named Entities (Named entities) and Normalization of a wider class of text spans, including the normalization of different parts of speech (Generic spans).The best solution has achieved 0.9645 accuracy on the Generic spans task and 0.9575 on the Named entities task.The presented solutions are in the public domain at https://github.com/RussianNLP/RuNormAS-solution