Using Generative Pretrained Transformer-3 Models for Russian News Clustering and Title Generation tasks
SberDevices, Sberbank, Moscow, Russia, Maria Tikhonova, Tatiana Shavrina, Dina Pisarevskaya, Oleh Shliazhko, SberDevices, Sberbank, Moscow, Russia · Computational Linguistics and Intellectual Technologies · 2021
The paper presents a methodology for news clustering and news headline generation based on the zero-shot approach and minimal tuning of the RuGPT-3 architecture (Generative Pretrained Transformer 3 for Russian).The solution is presented in a competition for news clustering, headline selection and generation.The following approaches are described: 1) zero-shot unsupervised classification based on pairwise news perplexity: the method requires no training or model fine-tuning and yields 0.7 F1-measure.2) fine-tuning: news headlines generation with the best result 0.292 ROUGE and 0.596 BLEU.