MAML-BERT in addressing low-resource text classification tasks
Yang Hu, Guiyun Zhang · 2024
The traditional approach for text classification tasks commonly involves fine-tuning existing general-purpose or base models. However, when training data is extremely scarce, this approach may lead to overfitting or getting stuck in local optima. This paper investigates meta-learning-based methods for low-resource text classification, taking a diabetes dataset as an example, and proposes a meta-learning framework combined with data augmentation techniques. Firstly, we utilize an abstract summarization method to effectively augment the original dataset, alleviating potential issues caused by data imbalance while enhancing the diversity and generalization of the samples. Then, by employing a meta-learning algorithm, we achieve optimization and adjustment of the global initialization parameters. Subsequently, these parameters will guide the fine-tuning of the pre-trained BERT model to adapt to the diabetes text classification task. Experimental results show that our method significantly improves classification performance under low-resource conditions, providing new insights for handling low-resource text classification problems in similar domains.