Efficient Stochastic Gradient Descent for Distributionally Robust Learning
Soumyadip Ghosh, Mark S. Squillante, Ebisa Wollega · arXiv (Cornell University) · 2018
We consider a new stochastic gradient descent algorithm for efficiently solving general min-max optimization problems that arise naturally in distributionally robust learning. By focusing on the entire dataset, current approaches do not scale well. We address this issue by initially focusing on a subset of the data and progressively increasing this support to statistically cover the entire dataset.