A Min-plus-SDDP Algorithm for Deterministic Multistage Convex Programming

We consider discrete time optimal control problems with finite horizon involving continuous states and possibly both continuous and discrete controls, subject to non-stationary linear dynamics and convex costs. In this general framework, we present a stochastic algorithm which generates monotone approximations of the value functions as a pointwise supremum or infimum of basic functions (for example affine or quadratic) which are randomly selected. We give sufficient conditions on the way basic functions are selected in order to ensure almost sure convergence of the approximations to the value function on a set of interest. Then we study a linear-quadratic optimal control problem with one control constraint. On this toy example we show how to use our algorithm in order to build lower approximations, like the SDDP algorithm, as supremum of affine cuts and upper approximations, by min-plus techniques, as infimum of quadratic fonctions.