Using data resampling and category weight adjustment to solve sample imbalance
Zimeng Leng · Applied and Computational Engineering · 2024
The purpose of this thesis is to investigate the application of artificial intelligence and machine learning in solving the sample imbalance problem. The sample imbalance problem refers to the phenomenon that the number of different categories of samples in the training data varies greatly, resulting in the poor performance of traditional machine learning algorithms on a few categories of samples. To address this problem, this paper proposes a new approach combining data resampling and category weight adjustment strategies. First, the sample distribution of the dataset is adjusted by undersampling and oversampling techniques to balance the number of samples from different categories. Then, during the model training process, different weights are assigned to the samples of different categories so that the model pays more attention to the samples of a few categories. The experimental results show that the method achieves significant performance improvement on multiple datasets. In addition, this paper compares other commonly used methods for solving the sample imbalance problem and analyzes and discusses them in detail. Finally, this study offers a practical solution to the problem of sample imbalance and provides guidance for research in related fields.