Extrinsic evaluation of UDPipe pre-trained models in application to hyper-hyponymic verbal pairs extraction from dictionary definitions
Oksana I. Antropova, Ekaterina A. Ogorodnikova · AIP conference proceedings · 2020
Our ongoing research is dedicated to the development of a method of computer-aided hyper-hyponymic verbal pairs extraction from dictionary definitions. Syntactic signals provide important information for the task. The main advantage of syntax-based method is that it allows one to distinguish hypernyms from synonyms, which is very difficult for distributional methods such as word embeddings as long as hypernyms and synonyms both tend to occur in similar contexts. This paper focuses on extrinsic evaluation of all three UDPipe pre-trained models for the Russian language. It is shown that despite high performance on different corpora the models’ perform poorly on our highly specific corpus consisting of dictionary definitions of verbs.