TALES: Test Set of Portuguese Lexical-Semantic Relations for Assessing Word Embeddings
Hugo Gonçalo Oliveira, Tiago Sousa, Ana Flávia Elói da Silva Alves · European Conference on Artificial Intelligence · 2020
Static word embeddings, like word2vec or GloVe, are of- ten assessed when solving syntactic and semantic analogies. Among the latter, we are interested in relations that one would find in lexical- semantic knowledge bases like WordNet, also covered in analogy test sets for English. This paper describes the creation of a new test for assessing Portuguese word embeddings, dubbed TALES, with an exclusive focus on lexical-semantic relations, acquired from lexi- cal resources in Portuguese. It further reports on the performance of methods previously used for solving analogies, with pre-trained Por- tuguese word embeddings, when applied to the created dataset, an experiment that revealed that TALES is challenging to solve. Results achieved are briefly discussed, with conclusions that may be useful for developing new approaches for this problem, possibly new em- beddings, as well as future versions of TALES.