Ensemble-based Fine-Tuning Strategy for Temporal Relation Extraction from the Clinical Narrative
Lijing Wang, Timothy Miller, Steven J. Bethard, Guergana Savova · 2022
In this paper, we investigate ensemble methods for fine-tuning transformer-based pretrained models for clinical natural language processing tasks, specifically temporal relation extraction from the clinical narrative.Our experimental results on the THYME data show that ensembling as a fine-tuning strategy can further boost model performance over single learners optimized for hyperparameters.Dynamic snapshot ensembling is particularly beneficial as it finetunes a wide array of parameters and results in a 2.8% absolute improvement in F1 over the base single learner.