David Parker
Professor of Computer Science, University of Birmingham
[BGP21] Edoardo Bacci, Mirco Gia­cob­be, David Parker. Verifying Reinforcement Learning up to Infinity. In Proc. 30th International Joint Conference on Artificial Intelligence (IJCAI'21). To appear. August 2021. [pdf] [bib]
Downloads:  pdf pdf (1.1 MB)  bib bib
Notes: Source code is available online: https://github.com/phate09/SafeRL Infinity.
Abstract. Formally verifying that reinforcement learning systems act safely is increasingly important, but existing methods only verify over finite time. This is of limited use for dynamical systems that run indefinitely. We introduce the first method for verifying the time-unbounded safety of neural networks controlling dynamical systems. We develop a novel abstract interpretation method which, by constructing adaptable template-based polyhedra using MILP and interval arithmetic, yields sound—safe and invariant—overapproximations of the reach set. This provides stronger safety guarantees than previous time-bounded methods and shows whether the agent has generalised beyond the length of its training episodes. Our method supports ReLU activation functions and systems with linear, piecewise-linear and non-linear dynamics defined with polynomial and transcendental functions. We demonstrate its efficacy on a range of benchmark control problems.