Benchmarking Meta-embeddings: What Works and What Does Not
Iker García-Ferrero, Rodrigo Agerri, Germán Rigau · 2021
In the last few years, several methods have been proposed to build meta-embeddings.The general aim was to obtain new representations integrating complementary knowledge from different source pre-trained embeddings thereby improving their overall quality.However, previous meta-embeddings have been evaluated using a variety of methods and datasets, which makes it difficult to draw meaningful conclusions regarding the merits of each approach.In this paper we propose a unified common framework, including both intrinsic and extrinsic tasks, for a fair and objective meta-embeddings evaluation.Furthermore, we present a new method to generate meta-embeddings, outperforming previous work on a large number of intrinsic evaluation benchmarks.Our evaluation framework also allows us to conclude that previous extrinsic evaluations of meta-embeddings have been overestimated. 1