EU-BERT: Efficient Unified Named Entity Recognition via Multi-exit BERT
Xinyu Liu · 2023
Named entity recognition (NER) is an important Natural Language Processing (NLP) task with wide application, such as document analysis, knowledge graph and query understanding. Although pretrain-based language models such as BERT has achieved great results on NER task, the large amount of parameters of BERT makes it slow during inference, which limits its usage in industry. Experiments show that the metric of traditional early exiting has great defects. In this work, we propose EU-BERT, an Efficient Unified named entity recognition framework via multi-exit BERT, to accelerate BERT inference on NER task. To handle nested and discontinuous NER tasks, we adopt W2NER [1], a unified NER framework using table filling technique. EU-BERT proposes a better metric for early exiting and uses contrastive learning to enhance its ability. Experiments on 9 benchmark NER datasets demonstrate that our method can improve the performance of multi-exit BERT on NER task while maintaining its inference speed.