Decoding Domain-Specific NER: A Performance Evaluation of ChatGPT, Bi-LSTM, and BERT
Khushnood Adil Rafique, Maulik Pansuriya, Frank Wawrzik, Christoph Grimm · 2024
Named Entity Recognition (NER) is a vital method in natural language processing (NLP), extracting information by identifying and categorizing entities in textual data. This study delves into the dynamic landscape of NER models, considering recent advances in transfer learning with transformers like Bidirectional Encoder Representations from Transformers (BERT) and large language models such as ChatGPT. We evaluate and compare their performances on a custom domain-specific and limited corpus. Utilizing state-of-the-art techniques, we model multi-layered prompts for ChatGPT, concurrently utilizing the Bidirectional Long Short-Term Memory Networks (Bi-LSTM) model and fine-tuning a pre-trained BERT model for NER on the corpus. Our study offers crucial insights into these models’ viability under constrained, domain-specific dataset conditions, given the effectiveness of transfer learning and zeroshot classification with large language models. Our experimental results highlight that Bi-LSTM excels over ChatGPT and finetuned BERT on the custom dataset, with the added benefit of faster training times. We emphasize the importance of considering domain and task-specific factors and corpus size when selecting a model.