Matching semantic sketches to predicates in context using the BERT model
NRU HSE Moscow, Russia, Polina Aleksandrova, Anna Mokhova, NRU HSE Moscow, Russia, Maria Nikolaenkova, NRU HSE Moscow, Russia · Computational Linguistics and Intellectual Technologies · 2021
Modern language models have extensive information about the compatibility and meanings of various words.One of the ways to represent such lexical information, which is presented in the present study, is the construction of semantic sketches.This paper presents a solution to the task of predicting a predicate from its most frequent actants and sirconstants using the application of the BERT neural network, which showed the best quality metrics in the Dialogue Evaluation SemSketches competition.The study analyzed several solutions approaching this task and ways to improve them based on the peculiarities of the architecture and the nature of data in terms of linguistics.The results of testing the selected methods showed that the most successful tool for determining the semantic sketch of a predicate is the Conversational RuBERT model combined with the search for synonyms of the verbs sought in the training data.Other promising ways to improve the quality of mapping the predicate to its semantic sketch include the use of contextualized embeddings to be able to take context into account, as well as fine-tuning of the models used.