Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Umut Simsekli; Alain Durmus; Roland Badeau; Gael Richard; Eric Moulines; A. Taylan Cemgil

doi:10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Umut Simsekli
, Alain Durmus
, Roland Badeau
, Gael Richard
, Eric Moulines
, A. Taylan Cemgil

Université Paris-Saclay
Department of Computer Engineering
Bogazici University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Citations (Scopus)

Abstract

Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

Original language	English
Title of host publication	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2242-2246
Number of pages	5
ISBN (Electronic)	9781509041176
DOIs	https://doi.org/10.1109/ICASSP.2017.7952555
Publication status	Published - 16 Jun 2017
Event	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - New Orleans, United States Duration: 5 Mar 2017 → 9 Mar 2017

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
ISSN (Print)	1520-6149

Conference

Conference	2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017
Country/Territory	United States
City	New Orleans
Period	5/03/17 → 9/03/17

Keywords

Beta Divergence
Non-Negative Matrix Factorization
Richardson-Romberg Extrapolation
Stochastic Gradient MCMC
Tweedie Distribution

Access to Document

10.1109/ICASSP.2017.7952555

Cite this

Simsekli, U., Durmus, A., Badeau, R., Richard, G., Moulines, E., & Cemgil, A. T. (2017). Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. In 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings (pp. 2242-2246). Article 7952555 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICASSP.2017.7952555

Simsekli, Umut ; Durmus, Alain ; Badeau, Roland et al. / Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 2242-2246 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

@inproceedings{ef14db8fffb2438db6faff1279e8b27a,

title = "Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization",

abstract = "Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.",

keywords = "Beta Divergence, Non-Negative Matrix Factorization, Richardson-Romberg Extrapolation, Stochastic Gradient MCMC, Tweedie Distribution",

author = "Umut Simsekli and Alain Durmus and Roland Badeau and Gael Richard and Eric Moulines and Cemgil, \{A. Taylan\}",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 ; Conference date: 05-03-2017 Through 09-03-2017",

year = "2017",

month = jun,

day = "16",

doi = "10.1109/ICASSP.2017.7952555",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2242--2246",

booktitle = "2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings",

}

Simsekli, U, Durmus, A , Badeau, R , Richard, G , Moulines, E & Cemgil, AT 2017, Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. in 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings., 7952555, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 2242-2246, 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017, New Orleans, United States, 5/03/17. https://doi.org/10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. / Simsekli, Umut; Durmus, Alain ; Badeau, Roland et al.
2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2242-2246 7952555 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

AU - Simsekli, Umut

AU - Durmus, Alain

AU - Badeau, Roland

AU - Richard, Gael

AU - Moulines, Eric

AU - Cemgil, A. Taylan

PY - 2017/6/16

Y1 - 2017/6/16

N2 - Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

AB - Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods have become popular in modern data analysis problems due to their computational efficiency. Even though they have proved useful for many statistical models, the application of SG-MCMC to non-negative matrix factorization (NMF) models has not yet been extensively explored. In this study, we develop two parallel SG-MCMC algorithms for a broad range of NMF models. We exploit the conditional independence structure of the NMF models and utilize a stratified sub-sampling approach for enabling parallelization. We illustrate the proposed algorithms on an image restoration task and report encouraging results.

KW - Beta Divergence

KW - Non-Negative Matrix Factorization

KW - Richardson-Romberg Extrapolation

KW - Stochastic Gradient MCMC

KW - Tweedie Distribution

U2 - 10.1109/ICASSP.2017.7952555

DO - 10.1109/ICASSP.2017.7952555

M3 - Conference contribution

AN - SCOPUS:85023757502

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 2242

EP - 2246

BT - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017

Y2 - 5 March 2017 through 9 March 2017

ER -

Simsekli U, Durmus A , Badeau R , Richard G , Moulines E, Cemgil AT. Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization. In 2017 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2242-2246. 7952555. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP.2017.7952555

Parallelized Stochastic Gradient Markov Chain Monte Carlo algorithms for non-negative matrix factorization

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this