On Convergence and Generalization of Dropout Training
Poorya Mianjy, Raman Arora · arXiv (Cornell University) · 2020
We study dropout in two-layer neural networks with rectified linear unit (ReLU) activations. Under mild overparametrization and assuming that the limiting kernel can separate the data distribution with a positive margin, we show that dropout training with logistic loss achieves $ε$-suboptimality in test error in $O(1/ε)$ iterations.