TY - JOUR
T1 - On Stochastic Gradient Langevin Dynamics with Dependent Data Streams
T2 - The Fully Nonconvex Case
AU - Chau, Ngoc Huy
AU - Moulines, Éric
AU - Rásonyi, Miklós
AU - Sabanis, Sotirios
AU - Zhang, Ying
N1 - Publisher Copyright:
© 2021 Society for Industrial and Applied Mathematics.
PY - 2021/1/1
Y1 - 2021/1/1
N2 - We consider the problem of sampling from a target distribution, which is not necessarily log-concave, in the context of empirical risk minimization and stochastic optimization as presented in [M. Raginsky, A. Rakhlin, and M. Telgarsky, Proc. Mach. Learn. Res., 65 (2017), pp. 1674–1703]. Non-asymptotic results are established in the L1-Wasserstein distance for the behavior of stochastic gradient Langevin dynamics algorithms. We allow gradient estimates based on dependent data streams. Our convergence estimates are sharper and uniform in the number of iterations, in contrast to those in previous studies.
AB - We consider the problem of sampling from a target distribution, which is not necessarily log-concave, in the context of empirical risk minimization and stochastic optimization as presented in [M. Raginsky, A. Rakhlin, and M. Telgarsky, Proc. Mach. Learn. Res., 65 (2017), pp. 1674–1703]. Non-asymptotic results are established in the L1-Wasserstein distance for the behavior of stochastic gradient Langevin dynamics algorithms. We allow gradient estimates based on dependent data streams. Our convergence estimates are sharper and uniform in the number of iterations, in contrast to those in previous studies.
KW - Langevin dynamics
KW - contraction
KW - convergence guarantees
KW - nonconvex optimization
KW - stochastic gradient
UR - https://www.scopus.com/pages/publications/105010025290
U2 - 10.1137/20M1355392
DO - 10.1137/20M1355392
M3 - Article
AN - SCOPUS:105010025290
SN - 2577-0187
VL - 3
SP - 959
EP - 986
JO - SIAM Journal on Mathematics of Data Science
JF - SIAM Journal on Mathematics of Data Science
IS - 3
ER -