Embedding a heavy-ball type of momentum into the estimating sequences
Endrit Dosti, Sergiy A. Vorobyov, Themistoklis Charalambous · Signal Processing · 2025
We present a new accelerated gradient-based method for solving smooth unconstrained optimization problems. The new method exploits additional information about the objective function and is built by embedding a heavy-ball type of momentum into the Fast Gradient Method (FGM). We devise a generalization of the estimating sequences, which allows for encoding any form of information about the objective function that can aid in further accelerating the minimization process. In the black box framework, we propose a construction for the generalized estimating sequences, which is obtained by exploiting the history of the previously constructed estimating functions. Moreover, we prove that the proposed method requires at most κ 2 ln 1 ϵ + O ( 1 ) iterations to find a point x with f ( x ) − f ∗ ≤ ϵ , where ϵ is the desired tolerance and κ is the condition number of the problem. Our theoretical results are corroborated by numerical experiments on various types of optimization problems, often dealt with in different areas of the information processing sciences. Both synthetic and real-world datasets are utilized to demonstrate the efficiency of our proposed method in terms of decreasing the distance to the optimal solution, the norm of the gradient and the function value.