Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Gersende Fort; Eric Moulines; Hoi To Wai

doi:10.1109/ICASSP39728.2021.9414271

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Université de Toulouse
The Chinese University of Hong Kong

Résultats de recherche: Contribution à un journal › Article de conférence › Revue par des pairs

Résumé

The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

langue originale	Anglais
Pages (de - à)	3135-3139
Nombre de pages	5
journal	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2021-June
Les DOIs	https://doi.org/10.1109/ICASSP39728.2021.9414271
état	Publié - 1 janv. 2021
Evénement	2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 - Virtual, Toronto, Canada Durée: 6 juin 2021 → 11 juin 2021

Accès au document

10.1109/ICASSP39728.2021.9414271

Contient cette citation

@article{99f4a12ff3934ac9a6eb65eaba8ef341,

title = "Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization",

abstract = "The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.",

keywords = "Expectation Maximization, Large scale learning, Latent variable analysis, Nonconvex stochastic optimization, Variance reduction",

author = "Gersende Fort and Eric Moulines and Wai, \{Hoi To\}",

note = "Publisher Copyright: {\textcopyright}2021 IEEE; 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021 ; Conference date: 06-06-2021 Through 11-06-2021",

year = "2021",

month = jan,

day = "1",

doi = "10.1109/ICASSP39728.2021.9414271",

language = "English",

volume = "2021-June",

pages = "3135--3139",

journal = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

issn = "1520-6149",

}

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization. / Fort, Gersende ; Moulines, Eric; Wai, Hoi To.
Dans: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Vol 2021-June, 01.01.2021, p. 3135-3139.

Résultats de recherche: Contribution à un journal › Article de conférence › Revue par des pairs

TY - JOUR

T1 - Geom-Spider-EM

T2 - 2021 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2021

AU - Fort, Gersende

AU - Moulines, Eric

AU - Wai, Hoi To

PY - 2021/1/1

Y1 - 2021/1/1

N2 - The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

AB - The Expectation Maximization (EM) algorithm is a key reference for inference in latent variable models; unfortunately, its computational cost is prohibitive in the large scale learning setting. In this paper, we propose an extension of the Stochastic Path-Integrated Differential EstimatoR EM (SPIDER-EM) and derive complexity bounds for this novel algorithm, designed to solve smooth nonconvex finite-sum optimization problems. We show that it reaches the same state of the art complexity bounds as SPIDER-EM; and provide conditions for a linear rate of convergence. Numerical results support our findings.

KW - Expectation Maximization

KW - Large scale learning

KW - Latent variable analysis

KW - Nonconvex stochastic optimization

KW - Variance reduction

U2 - 10.1109/ICASSP39728.2021.9414271

DO - 10.1109/ICASSP39728.2021.9414271

M3 - Conference article

AN - SCOPUS:85108132133

SN - 1520-6149

VL - 2021-June

SP - 3135

EP - 3139

JO - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

JF - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Y2 - 6 June 2021 through 11 June 2021

ER -

Geom-Spider-EM: Faster variance reduced stochastic expectation maximization for nonconvex finite-sum optimization

Résumé

Accès au document

Empreinte digitale

Contient cette citation