ETNLP: A Visual-Aided Systematic Approach to Select Pre-Trained Embeddings for a Downstream Task
Xuan-Son Vu, Thanh Vu, Son Ngoc Tran, Lili Jiang · 2019
Given many recent advanced embedding models, selecting pre-trained word embedding (a.k.a., word representation) models best fit for a specific downstream task is non-trivial.In this paper, we propose a systematic approach, called ETNLP, for extracting, evaluating, and visualizing multiple sets of pre-trained word embeddings to determine which embeddings should be used in a downstream task.We demonstrate the effectiveness of the proposed approach on our pre-trained word embedding models in Vietnamese to select which models are suitable for a named entity recognition (NER) task.Specifically, we create a large Vietnamese word analogy list to evaluate and select the pre-trained embedding models for the task.We then utilize the selected embeddings for the NER task and achieve the new state-of-the-art results on the task benchmark dataset.We also apply the approach to another downstream task of privacy-guaranteed embedding selection, and show that it helps users quickly select the most suitable embeddings.In addition, we create an open-source system using the proposed systematic approach to facilitate similar studies on other NLP tasks.The source code and data are available at https: //github.com/vietnlp/etnlp.