The Global Landscape of Neural Networks: An Overview
Ruoyu Sun, Dawei Li, Shiyu Liang, Tian Ding, R. Srikant · IEEE Signal Processing Magazine · 2020
One of the major concerns for neural network training is that the nonconvexity of the associated loss functions may cause a bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks.