Mixing Static Word Embeddings and RoBERTa for Spatial Role Labeling
Alaeddine Moussa, Sébastien Fournier, Khaoula Mahmoudi, Bernard Espinasse, Sami Faïz · Procedia Computer Science · 2022
Language model pretraining has yielded significant results in diverse natural language processing tasks. RoberTa, an efficient method for pretraining self-supervised NLP systems, is a good example. Our hypothesis in this paper is that the performance of Spatial Role Labeling (SpRL) can be improved by combining static word vectors and bags of features with RoberTa vectors. Furthermore, we show that our method is successful in several SpRL datasets.