Boosting Stochastic Newton Descent for Bigdata large scale classification
Roberto D’Ambrosio, Wafa Belhajali, Michel Barlaud · 2014
Efficient Bigdata classification requires low cost learning methods. A standard approach involves Stochastic Gradient Descent algorithm (SGD) for the minimization of the Hinge Loss in the primal space. Although complexity of Stochastic Gradient Descent is linear with the number of samples these method suffers from slow convergence. In order to cope with this issue, we propose here a Boosting Stochastic Newton Descent (BSND) method for minimization of any calibrated loss in the primal space. BSND approximates the inverse Hessian by the best low-rank approximation. We validate BSND by benchmarking it against several variants of the state-of-the-art SGD algorithm on the the large scale ImageNet and Higgs dataset. We provide further core optimization for fast convergence. The results on big data set: ImageNet and Higgs display that BSND improves significantly accuracy of the SGD baseline while being faster by orders of magnitude.