An Empirical Study of Memorization in NLP
Xiaosen Zheng, Jing Jiang · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2022
A recent study by Feldman (2020) proposed a long-tail theory to explain the memorization behavior of deep learning models.However, memorization has not been empirically verified in the context of NLP, a gap addressed by this work.In this paper, we use three different NLP tasks to check if the long-tail theory holds.Our experiments demonstrate that top-ranked memorized training instances are likely atypical, and removing the top-memorized training instances leads to a more serious drop in test accuracy compared with removing training instances randomly.Furthermore, we develop an attribution method to better understand why a training instance is memorized.We empirically show that our memorization attribution method is faithful and share our interesting finding that the top-memorized parts of a training instance tend to be features negatively correlated with the class label.