The Role of Regularization in Overparameterized Neural Networks
Siddhartha Satpathi, Harsh Gupta, Shiyu Liang, R. Srikant · 2020
In this paper, we consider gradient descent on a regularized loss function for training an overparametrized neural network. We model the algorithm as an ODE and show how overparameterization and regularization work together to provide the right tradeoff between training and generalization errors.