An algorithm for maximum likelihood estimation of hidden markov models with unknown state-tying

For speech recognition based on hidden Markov modeling, parameter-tying, which consists in constraining some of the parameters of the model to share the same value, has emerged as a standard practice. In this paper, an original algorithm is proposed that makes it possible to jointly estimate both the shared model parameters and the tying characteristics, using the maximum likelihood criterion. The proposed algorithm is based on a recently introduced extension of the classic expectation-maximization (EM) framework. The convergence properties of this class of algorithms are analyzed in detail. The method is evaluated on an isolated word recognition task using hidden Markov models (HMM's) with Gaussian observation densities and tying at the state level. Finally, the extension of this method to the case of mixture observation densities with tying at the mixture component level is discussed.