Question Answering on Biomedical Research Papers using Transfer Learning on BERT-Base Models
Shushanta Pudasaini, Subarna Shakya · 2023
Reading entire research papers for a simple piece of information is very exhausting. Likewise, finding and arranging research papers with a keyword is also exhausting. This paper aims to automate such exhausting processes while working with research papers. This automation is done using a Natural Language Processing (NLP) technique: Question Answering System. The paper introduces benchmark data for developing such a Question Answering system finetuned for biomedical research papers. The data is developed by the annotation of question, context, and answer pairs from biomedical research papers from Pubmed. Applying transfer learning to this benchmark data on several pretrained large language models such as BERT, RoBERTa, PubmedBERT and BioBERT, the paper introduces a state-of-the-art model for such biomedical question-answering data. The model developed by finetuning this custom data on PubmedBERT released by Microsoft achieved an Exact Match of 83.89 and an F-score of 89.67.