On Stochastic Gradient Langevin Dynamics with Dependent Data Streams: The Fully Nonconvex Case

We consider the problem of sampling from a target distribution, which is not necessarily log-concave, in the context of empirical risk minimization and stochastic optimization as presented in [M. Raginsky, A. Rakhlin, and M. Telgarsky, Proc. Mach. Learn. Res., 65 (2017), pp. 1674–1703]. Non-asymptotic results are established in the L¹-Wasserstein distance for the behavior of stochastic gradient Langevin dynamics algorithms. We allow gradient estimates based on dependent data streams. Our convergence estimates are sharper and uniform in the number of iterations, in contrast to those in previous studies.