Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning
Bryn Elesedy, Varun Kanade, Yee Whye Teh · arXiv (Cornell University) · 2020
We analyse the pruning procedure behind the lottery ticket hypothesis arXiv:1803.03635v5, iterative magnitude pruning (IMP), when applied to linear models trained by gradient flow. We begin by presenting sufficient conditions on the statistical structure of the features under which IMP prunes those features that have smallest projection onto the data. Following this, we explore IMP as a method for sparse estimation.