Harnessing Open-Source LLMs for Tender Named Entity Recognition
Asim Abbas, Venelin Kovatchev, Mark Lee, Niloofer Shanavas, Mubashir Ali · International conference Recent advances in natural language processing · 2025
In the public procurement domain, extracting accurate tender entities from unstructured text remains a critical, less explored challenge, because tender data is highly sensitive and confidential, and not available openly.Previously, state-of-the-art NLP models were developed for this task; however developing an NER model from scratch required huge amounts of data and resources.Similarly, performing fine-tuning of a transformer-based model like BERT requires training data, as a result posing challenges in training data cost, model generalization, and data privacy.To address these challenges, an emerging LLM such as GPT-4 in a Few-shot learning environment achieves SOTA performance comparable to fine-tuned models.However, being dependent on the closedsource commercial LLMs involves high cost and privacy concerns.In this study, we have investigated open-source LLMs like Mistral and LLAMA-3, focusing on the tender domain for the NER tasks on local consumer-grade CPUs in three different environments: Zeroshot, One-shot, and Few-shot learning.The motivation is to efficiently lessen costs compared to a cloud solution while preserving accuracy and data privacy.Similarly, we have utilized two datasets open-source from Singapore and closed-source commercially sensitive data provided by Siemens.As a result, all the open-source LLMs achieve above 85% F1score on an open-source dataset and above 90% F1-score on a closed-source dataset.