Neural collapse with unconstrained features
Dustin G. Mixon, Hans Parshall, Jianzong Pi · arXiv (Cornell University) · 2020
Neural collapse is an emergent phenomenon in deep learning that was recently discovered by Papyan, Han and Donoho. We propose a simple "unconstrained features model" in which neural collapse also emerges empirically. By studying this model, we provide some explanation for the emergence of neural collapse in terms of the landscape of empirical risk.