Adaptive Markov chain Monte Carlo: Theory and methods

Markov chain Monte Carlo (MCMC) methods allow us to generate samples from an arbitrary distribution π known up to a scaling factor; see [46]. The algorithm consists in sampling a Markov chain {X_k, k ≥ 0} on a state space X with transition probability P admitting π as its unique invariant distribution. In most MCMC algorithms known so far, the transition probability P of the Markov chain depends on some tuning parameter θ defined on some space Θ which can be either finite dimensional or infinite dimensional. The success of the MCMC procedure depends crucially upon a proper choice of θ.To illustrate, consider the standard Metropolis–Hastings (MH) algorithm. For simplicity, we assume that π has a density also denoted by π with respect to the Lebesgue measure on X = ℝ^dendowed with its Borel σ-field χ. Given that the chain is at x, a candidate y is sampled from a proposal transition density q(x, ·) and is accepted with probability α(x, y) defined as where min(a, b). Otherwise, the move is rejected and the Markov chain stays at its current location x A commonly used choice for the proposal kernel is the symmetric increment random walk leading to the random walk MH algorithm (hereafter SRWM), in which q(x, y) = q(y - x) for all (x, y) ∈ X × X, for some symmetric proposal density function q on X.