Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Gersende Fort; Eric Moulines; Hoi To Wai

doi:10.1109/ICASSP39728.2021.9414271

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Research output: Contribution to journal › Conference article › peer-review

Abstract

The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

Original language	English
Pages (from-to)	3135-3139
Number of pages	5
Journal	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2021-June
DOIs	https://doi.org/10.1109/ICASSP39728.2021.9414271
Publication status	Published - 1 Jan 2021
Event	2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada Duration: 6 Jun 2021 → 11 Jun 2021

Keywords

Expectation Maximization
Large scale learning
Latent variable analysis
Nonconvex stochastic optimization
Variance reduction

Access to Document

10.1109/ICASSP39728.2021.9414271

Cite this

@article{4873f4512a5a4d518ef1df79d857bde5,

title = "Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization",

abstract = "The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.",

keywords = "Expectation Maximization, Large scale learning, Latent variable analysis, Nonconvex stochastic optimization, Variance reduction",

author = "Gersende Fort and Eric Moulines and Wai, \{Hoi To\}",

note = "Publisher Copyright: {\textcopyright}2021 IEEE; 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 ; Conference date: 06-06-2021 Through 11-06-2021",

year = "2021",

month = jan,

day = "1",

doi = "10.1109/ICASSP39728.2021.9414271",

language = "English",

volume = "2021-June",

pages = "3135--3139",

journal = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

issn = "1520-6149",

}

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization. / Fort, Gersende ; Moulines, Eric; Wai, Hoi To.
In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Vol. 2021-June, 01.01.2021, p. 3135-3139.

Research output: Contribution to journal › Conference article › peer-review

TY - JOUR

T1 - Geom-Spider-EM

T2 - 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021

AU - Fort, Gersende

AU - Moulines, Eric

AU - Wai, Hoi To

PY - 2021/1/1

Y1 - 2021/1/1

N2 - The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

AB - The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

KW - Expectation Maximization

KW - Large scale learning

KW - Latent variable analysis

KW - Nonconvex stochastic optimization

KW - Variance reduction

U2 - 10.1109/ICASSP39728.2021.9414271

DO - 10.1109/ICASSP39728.2021.9414271

M3 - Conference article

AN - SCOPUS:85108132133

SN - 1520-6149

VL - 2021-June

SP - 3135

EP - 3139

JO - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

JF - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Y2 - 6 June 2021 through 11 June 2021

ER -

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Abstract

Keywords

Access to Document

Fingerprint

Cite this