A vanilla Rao-Blackwellization of Metropolis-Hastings algorithms

Casella and Robert [Biometrika 83 (1996) 81-94] presented a general Rao-Blackwellization principle for accept-reject and Metropolis-Hastings schemes that leads to significant decreases in the variance of the resulting estimators, but at a high cost in computation and storage. Adopting a completely different perspective, we introduce instead a universal scheme that guarantees variance reductions in all Metropolis-Hastings-based estimators while keeping the computation cost under control.We establish a central limit theorem for the improved estimators and illustrate their performances on toy examples and on a probit model estimation.