Probabilistic regressor chains with Monte Carlo methods

A large number and diversity of techniques have been offered in the literature in recent years for solving multi-label classification tasks, including classifier chains where predictions are cascaded to other models as additional features. Chaining methods have often providing state of the art results, and the idea of extending it to multi-output regression has already been trialed. However, these ‘regressor chains’ have seen limited applicability, on account of yielding relatively little predictive performance compared to individual regression models, and also limited interpretability. In this work we identify and discuss the main limitations of regressor chains, including an analysis of different base models, loss functions, explainability, and other desiderata of real-world applications. We develop and examine techniques to overcome these limitations. In particular we present Monte Carlo schemes in the framework of probabilistic chains. We show they can be effective, flexible and useful in different areas. Overall, we also place regressor chains in context among general multi-output learning with continuous outputs, and in doing this shed additional light on the applicability of chaining to machine learning tasks.