Scaling Analysis of Delayed Rejection MCMC Methods

In this paper, we study the asymptotic efficiency of the delayed rejection strategy. In particular, the efficiency of the delayed rejection Metropolis-Hastings algorithm is compared to that of the regular Metropolis algorithm. To allow for a fair comparison, the study is carried under optimal mixing conditions for each of these algorithms. After introducing optimal scaling results for the delayed rejection (DR) algorithm, we outline the fact that the second proposal after the first rejection is discarded, with a probability tending to 1 as the dimension of the target density increases. To overcome this drawback, a modification of the delayed rejection algorithm is proposed, in which the direction of the different proposals is fixed once for all, and the Metropolis-Hastings accept-reject mechanism is used to select a proper scaling along the search direction. It is shown that this strategy significantly outperforms the original DR and Metropolis algorithms, especially when the dimension becomes large. We include numerical studies to validate these conclusions.