LEA: Meta Knowledge-Driven Self-Attentive Document Embedding for Few-Shot Text Classification
Seung-Kyu Hong, Tae Young Jang · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · 2022
In recent years, NLP has advanced greatly along with the proliferation of pre-trained language models.The pre-trained language models are also properly adapted to downstream tasks when there is sufficient labeled data.However, in real-world applications, we often encounter the deficiency of labeled data.When only given a few instances for a new task, extracting task-aware features from a pre-trained language model regardless of the adaptation is a promising alternative.In the study, we propose a novel embedding transfer method, called LEA, for leveraging pre-trained language models with even only few-shot instances.LEA derives meta-level attention aspects using our new meta-learning framework.We evaluate our method on five text classification benchmark datasets.The results show that the novel method robustly provides the competitive performance compared to recent few-shot learning methods.