Deep Neural Networks, Universal Approximation, and Geometric Control

Paulo Tabuada

Professor & Vijay K. Dhir Chair in Engineering, Electrical and Computer Engineering Department, University of California, Los Angeles

Seminar Information

Seminar Series
Dynamic Systems & Controls

Seminar Date - Time
May 19, 2023, 3:00 pm
-
4 PM

Seminar Location
EBU II 479, Von Karman-Penner Seminar Room


Abstract

Deep neural networks have drastically changed the landscape of several engineering areas such as computer vision and natural language processing. Notwithstanding the widespread success of deep networks in these, and many other areas, it is still not well understood why deep neural networks work so well. In particular, the question of which functions can be learned by deep neural networks has remained unanswered. 

In this talk we give an answer to this question for deep residual neural networks, a class of deep networks that can be interpreted as the time discretization of nonlinear control systems. We will show that the ability of these networks to memorize training data can be expressed through the control theoretic notion of controllability which can be proved using geometric control techniques. We then add an additional ingredient, monotonicity, to conclude that deep residual networks can approximate, to arbitrary accuracy with respect to the uniform norm, any continuous function on a compact subset of n-dimensional Euclidean space by using at most n+1 neurons per layer. We will conclude the talk by showing how these results pave the way for the use of deep networks in the perception pipeline of autonomous systems while providing formal (and probability free) guarantees of stability and robustness.

Speaker Bio

Paulo Tabuada was born in Lisbon, Portugal, one year after the Carnation Revolution. He received his "Licenciatura" degree in Aerospace Engineering from Instituto Superior Tecnico, Lisbon, Portugal in 1998 and his Ph.D. degree in Electrical and Computer Engineering in 2002 from the Institute for Systems and Robotics, a private research institute associated with Instituto Superior Tecnico. Between January 2002 and July 2003 he was a postdoctoral researcher at the University of Pennsylvania. After spending three years at the University of Notre Dame, as an Assistant Professor, he joined the Electrical and Computer Engineering Department at the University of California, Los Angeles, where he currently is the Vijay K. Dhir Professor of Engineering.

Paulo Tabuada's contributions to control and cyber-physical systems have been recognized by multiple awards including the NSF CAREER award in 2005, the Donald P. Eckman award in 2009, the George S. Axelby award in 2011, the Antonio Ruberti Prize in 2015, the grade of fellow awarded by IEEE in 2017 and by IFAC in 2019. He has been program chair and general chair for several conferences in the areas of control and of cyber-physical systems such as NecSys, HSCC, and ICCPS. He currently serves as the chair of HSCC’s steering committee and served on the editorial board of the IEEE Embedded Systems Letters and the IEEE Transactions on Automatic Control.