The Impact of Decreasing Dataset Sizes on Frozen Layer Transfer Learning
Michael Zheng · Scholarly review . · 2021
Transfer learning is a machine learning training method where a model trained for one task gets trained a second time for another task that is usually related yet different from the original task. Applying transfer learning during machine learning training helps to decrease the training time and computational resources needed, as the model has already been pretrained when it was trained for the first task. Transfer learning also helps to compensate for problems like underfitting that might occur when a machine learning model is being trained on a small dataset. This study aims to identify correlations between the effectiveness of transfer learning, specifically frozen layer transfer learning, and the amount of data provided. We accomplish this by applying transfer learning to a machine learning model and feeding it gradually decreasing amounts of data from a data set and observing the model accuracy. We observe that as the amount of provided data decreases, frozen layer transfer learning becomes increasingly less effective.