Convergence of markovian stochastic approximation with discontinuous dynamics

This paper is devoted to the convergence analysis of stochastic approximation algorithms of the form θ_n+1 = θ_n + γ_n+1H_θn(X_n+1), where {θ_n, n ϵ ℕ} is an Rd-valued sequence, {γ_n, n ϵ N} is a deterministic stepsize sequence, and {X_n, n ϵ N} is a controlled Markov chain. We study the convergence under weak assumptions on smoothness-in-θ of the function θ → H_θ(x). It is usually assumed that this function is continuous for any x; in this work, we relax this condition. Our results are illustrated by considering stochastic approximation algorithms for (adaptive) quantile estimation and a penalized version of the vector quantization.