When dynamic programming meets Lyapunov theory: robust stability and improved near-optimality guarantees

Dynamic programming (DP) plays a major role in various fields including reinforcement learning, operations research, computer sciences and of course control engineering. DP allows to solve general optimal control problems in terms of dynamical systems and cost functions. When exploiting DP in a control engineering context, it is often essential to endow the closed-loop system with stability guarantees. In this talk, we will present general conditions rooted in Lyapunov theory under which deterministic systems whose inputs are generated by a DP algorithm (like value iteration or policy iteration) exhibit stability properties. Interestingly, we will see that the presented Lyapunov-based conditions can be exploited to analyse the near-optimality properties of the algorithm used to generate the control inputs, thereby improving existing near-optimality bounds of the DP literature.