TY - GEN
T1 - The Perturbed Prox-Preconditioned Spider Algorithm for EM-Based Large Scale Learning
AU - Fort, G.
AU - Moulines, E.
N1 - Publisher Copyright:
© 2021 IEEE.
PY - 2021/7/11
Y1 - 2021/7/11
N2 - Incremental Expectation Maximization (EM) algorithms were introduced to design EM for the large scale learning framework by avoiding the full data set to be processed at each iteration. Nevertheless, these algorithms all assume that the conditional expectations of the sufficient statistics are explicit. In this paper, we propose a novel algorithm named Perturbed Prox-Preconditioned SPIDER (3P-SPIDER), which builds on the Stochastic Path Integral Differential EstimatoR EM (SPIDER-EM) algorithm. The 3P-SPIDER algorithm addresses many intractabilities of the E-step of EM; it also deals with non-smooth regularization and convex constraint set. Numerical experiments show that 3P-SPIDER outperforms other incremental EM methods and discuss the role of some design parameters.
AB - Incremental Expectation Maximization (EM) algorithms were introduced to design EM for the large scale learning framework by avoiding the full data set to be processed at each iteration. Nevertheless, these algorithms all assume that the conditional expectations of the sufficient statistics are explicit. In this paper, we propose a novel algorithm named Perturbed Prox-Preconditioned SPIDER (3P-SPIDER), which builds on the Stochastic Path Integral Differential EstimatoR EM (SPIDER-EM) algorithm. The 3P-SPIDER algorithm addresses many intractabilities of the E-step of EM; it also deals with non-smooth regularization and convex constraint set. Numerical experiments show that 3P-SPIDER outperforms other incremental EM methods and discuss the role of some design parameters.
KW - Accelerated Stochastic Approximation
KW - Control Variates
KW - Expectation Maximization algorithm
KW - Finite-sum Optimization
KW - Large Scale Learning
KW - Statistical Learning
U2 - 10.1109/SSP49050.2021.9513769
DO - 10.1109/SSP49050.2021.9513769
M3 - Conference contribution
AN - SCOPUS:85113507240
T3 - IEEE Workshop on Statistical Signal Processing Proceedings
SP - 316
EP - 320
BT - 2021 IEEE Statistical Signal Processing Workshop, SSP 2021
PB - IEEE Computer Society
T2 - 21st IEEE Statistical Signal Processing Workshop, SSP 2021
Y2 - 11 July 2021 through 14 July 2021
ER -