MGDA/D: A Multiple Gradient Descent Algorithm Based on Decomposition
Yawen Zhou, Ke Xue, Chao Qian · Proceedings of the Genetic and Evolutionary Computation Conference Companion · 2025
Multi-objective optimization (MOO) aims to optimize multiple conflicting objectives simultaneously. Gradient-free methods are widely used in MOO problems. However, in some scenarios, such as multitask learning, gradient-free algorithms are not suitable due to the high dimensionality of the solution space. Thus, many gradient-based multi-objective optimization methods have been proposed to find the Pareto front effeciently. However, these methods suffer from the difficulty of escaping local optima. In this paper, we propose a multiple gradient descent algorithm based on decomposition (MGDA/D), which can easily escape from the local optima and find a set of good and diverse Pareto solutions. MGDA/D first decomposes a multi-objective optimization problem into a number of scalar optimization subproblems and optimizes them simultaneously by gradient descent. When the solution of a subproblem falls into local optima, MGDA/D uses information from its neighboring subproblems to update the solution to escape. Experimental results confirm that MGDA/D outperforms the state-of-the-art algorithms on both benchmark problems and multi-task learning applications.