TY - GEN
T1 - ODE methods for Markov chain stability with applications to MCMC
AU - Fort, G.
AU - Moulines, E.
AU - Meyn, S.
AU - Priouret, P.
PY - 2006/12/1
Y1 - 2006/12/1
N2 - Fluid limit techniques have become a central tool to analyze queueing networks over the last decade, with applications to performance analysis, simulation, and optimization.In this paper some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space, and the initial condition by a large constant. The resulting fluid limit is the solution of an ODE in "most" of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similar to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov Chain Monte Carlo.
AB - Fluid limit techniques have become a central tool to analyze queueing networks over the last decade, with applications to performance analysis, simulation, and optimization.In this paper some of these techniques are extended to a general class of skip-free Markov chains. As in the case of queueing models, a fluid approximation is obtained by scaling time, space, and the initial condition by a large constant. The resulting fluid limit is the solution of an ODE in "most" of the state space. Stability and finer ergodic properties for the stochastic model then follow from stability of the set of fluid limits. Moreover, similar to the queueing context where fluid models are routinely used to design control policies, the structure of the limiting ODE in this general setting provides an understanding of the dynamics of the Markov chain. These results are illustrated through application to Markov Chain Monte Carlo.
KW - Fluid limit stability
KW - Fluid limits for general state-space Markov chains
KW - Markov Chain Monte-Carlo
KW - Metropolis-hastings algorithm
UR - https://www.scopus.com/pages/publications/34748820528
U2 - 10.1145/1190095.1190149
DO - 10.1145/1190095.1190149
M3 - Conference contribution
AN - SCOPUS:34748820528
SN - 1595935045
SN - 9781595935045
T3 - ACM International Conference Proceeding Series
BT - Proceedings of VALUETOOLS
T2 - VALUETOOLS: 1st International Conference on Performance Evaluation Methodologies and Tools
Y2 - 11 October 2006 through 13 October 2006
ER -