Multiplicative Reweighting for Robust Neural Network Optimization
Noga Bar, Tomer Koren, Raja Giryes · SIAM Journal on Imaging Sciences · 2025
Abstract. Neural networks are widespread due to their powerful performance. Yet they degrade in the presence of noisy labels at training time. Inspired by the setting of learning with expert advice, where multiplicative weight (MW) updates were recently shown to be robust to moderate data corruptions in expert advice, we propose to use MW for reweighting examples during neural network optimization. We theoretically establish the convergence of our method when used with gradient descent and prove its advantages in one-dimensional cases. We then validate empirically our findings for the general case by showing that MW improves the accuracy of neural networks in the presence of label noise on CIFAR-10, CIFAR-100, and Clothing1M. We also show the impact of our approach on adversarial robustness.