From learned to new relations through generative models combined with relations clustering and few-shot learning
Dorina-Mihaela Tirsogoiu, Anca Nicoleta Marginean · 2023
From the release of BERT in 2018 to the newest large language model of OpenAI, GPT-4, the corpus size of the training data has increased to the order of trillions of tokens. However, the models’ capabilities to expand and generalize their acquired knowledge remain a challenge. This research aims to analyze the ability of models to generalize relation types for the task of relation extraction, by clustering them in categories and training the model on half of the relations in a cluster. The generalization capabilities will be tested against zero-shot, one-shot, and few-shot learning, introducing relations from the other half of the cluster, to analyze how much data a model needs to achieve satisfactory results on generating the unseen relation types.