An Extension of the Second Order Dynamical System that Models Nesterov’s Convex Gradient Method

Cristian Daniel Alecsa, Szilárd Csaba László, Titus Pinţa

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper we deal with a general second order continuous dynamical system associated to a convex minimization problem with a Fréchet differentiable objective function. We show that inertial algorithms, such as Nesterov’s algorithm, can be obtained via the natural explicit discretization from our dynamical system. Our dynamical system can be viewed as a perturbed version of the heavy ball method with vanishing damping, however the perturbation is made in the argument of the gradient of the objective function. This perturbation seems to have a smoothing effect for the energy error and eliminates the oscillations obtained for this error in the case of the heavy ball method with vanishing damping, as some numerical experiments show. We prove that the value of the objective function in a generated trajectory converges in order O(1 / t2) to the global minimum of the objective function. Moreover, we obtain that a trajectory generated by the dynamical system converges to a minimum point of the objective function.

Original languageEnglish
Pages (from-to)1687-1716
Number of pages30
JournalApplied Mathematics & Optimization
Volume84
Issue number2
DOIs
Publication statusPublished - 1 Oct 2021
Externally publishedYes

Keywords

  • Continuous second order dynamical system
  • Convergence rate
  • Convex optimization
  • Heavy ball method
  • Inertial algorithm

Fingerprint

Dive into the research topics of 'An Extension of the Second Order Dynamical System that Models Nesterov’s Convex Gradient Method'. Together they form a unique fingerprint.

Cite this