Probabilistic Filter and Smoother for Variational Inference of Bayesian Linear Dynamical Systems

Variational inference of a Bayesian linear dynamical system is a powerful method for estimating latent variable sequences and learning sparse dynamic models in domains ranging from neuroscience to audio processing. The hardest part of the method is inferring the model's latent variable sequence. Here, we propose a solution using matrix inversion lemmas to derive what may be considered as the Bayesian counterparts to the Kalman filter and smoother, which are particular forms of the forward-backward algorithm that have known properties of numerical stability and efficiency that lead to cost growing linear with time. Opposed to existing methods, we do not augment the model dimensionality, use Cholesky decompositions or inaccurate numerical matrix inversions. We provide mathematical proof and empirical evidence that the new algorithm respects parameter expected values to more accurately infer latent state statistics. An application to Bayesian frequency estimation of a stochastic sum of sinusoids model is presented and compared with state-of-the-art estimators.