A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling

The expectation–maximization (EM) algorithm is a powerful computational technique for maximum likelihood estimation in incomplete data models. When the expectation step cannot be performed in closed form, a stochastic approximation of EM (SAEM) can be used. The convergence of the SAEM toward critical points of the observed likelihood has been proved and its numerical efficiency has been demonstrated. However, sampling from the posterior distribution may be intractable or have a high computational cost. Moreover, despite appealing features, the limit position of this algorithm can strongly depend on its starting one. Sampling from an approximation of the distribution in the expectation phase of the SAEM allows coping with these two issues. This new procedure is referred to as approximated-SAEM and is proved to converge toward critical points of the observed likelihood. Experiments on synthetic and real data highlight the performance of this algorithm in comparison to the SAEM and the EM when feasible.