Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Umut Simsekli; Alain Durmus; Roland Badeau; Gael Richard; Eric Moulines; A. Taylan Cemgil

doi:10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Umut Simsekli
, Alain Durmus
, Roland Badeau
, Gael Richard
, Eric Moulines
, A. Taylan Cemgil

Université Paris-Saclay
Bogazici University

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

Résumé

Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

langue originale	Anglais
titre	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings
Editeur	Institute of Electrical and Electronics Engineers Inc.
Pages	2242-2246
Nombre de pages	5
ISBN (Electronique)	9781509041176
Les DOIs	https://doi.org/10.1109/ICASSP.2017.7952555
état	Publié - 16 juin 2017
Evénement	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - New Orleans, États-Unis Durée: 5 mars 2017 → 9 mars 2017

Série de publications

Nom	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
ISSN (imprimé)	1520-6149

Une conférence

Une conférence	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017
Pays/Territoire	États-Unis
La ville	New Orleans
période	5/03/17 → 9/03/17

Accès au document

10.1109/ICASSP.2017.7952555

Contient cette citation

Simsekli, U., Durmus, A., Badeau, R., Richard, G., Moulines, E., & Cemgil, A. T. (2017). Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. Dans 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings (p. 2242-2246). Article 7952555 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2017.7952555

Simsekli, Umut ; Durmus, Alain ; Badeau, Roland et al. / Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2242-2246 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{ef14db8fffb2438db6faff1279e8b27a,

title = "Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization",

abstract = "Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.",

keywords = "Beta Divergence, Non-Negative Matrix Factorization, Richardson-Romberg Extrapolation, Stochastic Gradient MCMC, Tweedie Distribution",

author = "Umut Simsekli and Alain Durmus and Roland Badeau and Gael Richard and Eric Moulines and Cemgil, \{A. Taylan\}",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 ; Conference date: 05-03-2017 Through 09-03-2017",

year = "2017",

month = jun,

day = "16",

doi = "10.1109/ICASSP.2017.7952555",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2242--2246",

booktitle = "2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings",

}

Simsekli, U, Durmus, A , Badeau, R , Richard, G , Moulines, E & Cemgil, AT 2017, Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. Dans 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings., 7952555, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Institute of Electrical and Electronics Engineers Inc., p. 2242-2246, 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017, New Orleans, États-Unis, 5/03/17. https://doi.org/10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. / Simsekli, Umut; Durmus, Alain ; Badeau, Roland et al.
2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2242-2246 7952555 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

Résultats de recherche: Le chapitre dans un livre, un rapport, une anthologie ou une collection › Contribution à une conférence › Revue par des pairs

TY - GEN

T1 - Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

AU - Simsekli, Umut

AU - Durmus, Alain

AU - Badeau, Roland

AU - Richard, Gael

AU - Moulines, Eric

AU - Cemgil, A. Taylan

PY - 2017/6/16

Y1 - 2017/6/16

N2 - Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

AB - Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

KW - Beta Divergence

KW - Non-Negative Matrix Factorization

KW - Richardson-Romberg Extrapolation

KW - Stochastic Gradient MCMC

KW - Tweedie Distribution

U2 - 10.1109/ICASSP.2017.7952555

DO - 10.1109/ICASSP.2017.7952555

M3 - Conference contribution

AN - SCOPUS:85023757502

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 2242

EP - 2246

BT - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017

Y2 - 5 March 2017 through 9 March 2017

ER -

Simsekli U, Durmus A , Badeau R , Richard G , Moulines E, Cemgil AT. Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. Dans 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2242-2246. 7952555. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Résumé

Série de publications

Une conférence

Accès au document

Empreinte digitale

Contient cette citation