Investigating Overparameterization for Non-Negative Matrix Factorization in Collaborative Filtering
Yuhi Kawakami, Mahito Sugiyama · 2021
Overparameterization is one of the key techniques in modern machine learning, where a model with the higher complexity can generalize better on test data against the common knowledge of the bias-variance trade-off in classical statistical learning theory. In this paper, we empirically investigate the effect of overparameterization for matrix factorization-based models in collaborative filtering. Surprisingly, we firstly show that the performance of overparameterized non-negative matrix factorization (NMF) on test data gets better than that of the underparameterized NMF, which is commonly used to date, and is even competitive with the state-of-the-art collaborative filtering techniques. Moreover, we also show that the double descent phenomenon occurs when we increase the number of parameters of the NMF, where the test error decreases, increases, and decreases again as the model complexity grows, which has been recently reported in various machine learning methods such as deep learning models and kernel methods.