First-Order Constrained Optimization: Non-smooth Dynamical System Viewpoint

Sholom Schechtman; Daniil Tiapkin; Eric Moulines; Michael I. Jordan; Michael Muehlebach

doi:10.1016/j.ifacol.2022.09.030

First-Order Constrained Optimization: Non-smooth Dynamical System Viewpoint

Sholom Schechtman
, Daniil Tiapkin
, Eric Moulines
, Michael I. Jordan
, Michael Muehlebach

Ecole Polytechnique
National Research University
University of California, Berkeley
MPI-IS

Research output: Contribution to journal › Conference article › peer-review

Abstract

In a recent paper, Muehlebach and Jordan (2021a) proposed a novel algorithm for constrained optimization that uses original ideals from nonsmooth dynamical systems. In this work, we extend Muehlebach and Jordan (2021a) in several important directions: (i) we provide existence and convergence results for continuous-time trajectories under general conditions, and (ii) we provide a convergence guarantee for a perturbed version of the discrete-time version of the algorithm (covering stochastic gradient updates), for nonconvex and nonsmooth objective functions. Our analysis framework rationalizes the continuous-time and discrete-time cases, which not only provides an important intuition but could also enable convergence proofs for accelerated or Newton-like versions of our algorithm.

Original language	English
Pages (from-to)	236-241
Number of pages	6
Journal	IFAC-PapersOnLine
Volume	55
Issue number	16
DOIs	https://doi.org/10.1016/j.ifacol.2022.09.030
Publication status	Published - 1 Jul 2022
Externally published	Yes
Event	18th IFAC Workshop on Control Applications of Optimization, CAO 2022 - Gif sur Yvette, France Duration: 18 Jul 2022 → 22 Jul 2022

Keywords

Large scale optimization problems
Model predictive and optimization-based control
Static optimization problems

Access to Document

10.1016/j.ifacol.2022.09.030

Cite this

@article{f86beb8bcf7b4a9eb3f924fa0987d04c,

title = "First-Order Constrained Optimization: Non-smooth Dynamical System Viewpoint",

abstract = "In a recent paper, Muehlebach and Jordan (2021a) proposed a novel algorithm for constrained optimization that uses original ideals from nonsmooth dynamical systems. In this work, we extend Muehlebach and Jordan (2021a) in several important directions: (i) we provide existence and convergence results for continuous-time trajectories under general conditions, and (ii) we provide a convergence guarantee for a perturbed version of the discrete-time version of the algorithm (covering stochastic gradient updates), for nonconvex and nonsmooth objective functions. Our analysis framework rationalizes the continuous-time and discrete-time cases, which not only provides an important intuition but could also enable convergence proofs for accelerated or Newton-like versions of our algorithm.",

keywords = "Large scale optimization problems, Model predictive and optimization-based control, Static optimization problems",

author = "Sholom Schechtman and Daniil Tiapkin and Eric Moulines and Jordan, \{Michael I.\} and Michael Muehlebach",

note = "Publisher Copyright: Copyright {\textcopyright} 2022 The Authors.; 18th IFAC Workshop on Control Applications of Optimization, CAO 2022 ; Conference date: 18-07-2022 Through 22-07-2022",

year = "2022",

month = jul,

day = "1",

doi = "10.1016/j.ifacol.2022.09.030",

language = "English",

volume = "55",

pages = "236--241",

journal = "IFAC-PapersOnLine",

issn = "2405-8963",

number = "16",

}

TY - JOUR

T1 - First-Order Constrained Optimization

T2 - 18th IFAC Workshop on Control Applications of Optimization, CAO 2022

AU - Schechtman, Sholom

AU - Tiapkin, Daniil

AU - Moulines, Eric

AU - Jordan, Michael I.

AU - Muehlebach, Michael

PY - 2022/7/1

Y1 - 2022/7/1

N2 - In a recent paper, Muehlebach and Jordan (2021a) proposed a novel algorithm for constrained optimization that uses original ideals from nonsmooth dynamical systems. In this work, we extend Muehlebach and Jordan (2021a) in several important directions: (i) we provide existence and convergence results for continuous-time trajectories under general conditions, and (ii) we provide a convergence guarantee for a perturbed version of the discrete-time version of the algorithm (covering stochastic gradient updates), for nonconvex and nonsmooth objective functions. Our analysis framework rationalizes the continuous-time and discrete-time cases, which not only provides an important intuition but could also enable convergence proofs for accelerated or Newton-like versions of our algorithm.

AB - In a recent paper, Muehlebach and Jordan (2021a) proposed a novel algorithm for constrained optimization that uses original ideals from nonsmooth dynamical systems. In this work, we extend Muehlebach and Jordan (2021a) in several important directions: (i) we provide existence and convergence results for continuous-time trajectories under general conditions, and (ii) we provide a convergence guarantee for a perturbed version of the discrete-time version of the algorithm (covering stochastic gradient updates), for nonconvex and nonsmooth objective functions. Our analysis framework rationalizes the continuous-time and discrete-time cases, which not only provides an important intuition but could also enable convergence proofs for accelerated or Newton-like versions of our algorithm.

KW - Large scale optimization problems

KW - Model predictive and optimization-based control

KW - Static optimization problems

U2 - 10.1016/j.ifacol.2022.09.030

DO - 10.1016/j.ifacol.2022.09.030

M3 - Conference article

AN - SCOPUS:85142215078

SN - 2405-8963

VL - 55

SP - 236

EP - 241

JO - IFAC-PapersOnLine

JF - IFAC-PapersOnLine

IS - 16

Y2 - 18 July 2022 through 22 July 2022

ER -

First-Order Constrained Optimization: Non-smooth Dynamical System Viewpoint

Abstract

Keywords

Access to Document

Fingerprint

Cite this