Dynamic Systems & Controls

The desire to optimize performance is pervasive across engineering disciplines - and society for that matter – and the advent of ‘big data’ and learning-based methods have brought leaps in performance in many areas.

In this talk, I will build the connection between Hamilton-Jacobi-Bellman equations (HJB) and the multi-armed bandit (MAB) problems. HJB is an important equation in solving stochastic optimal control problems.

Many approaches to machine learning have struggled with applications that possess complex process dynamics. In contrast, human intelligence is adapted, and - - arguably - built to deal with complex dynamics.

We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step.

Deploying massive swarms of robots to solve real-world problems has been a research promise for over 50 years, but even today we still do not see clear paths for how an engineer should design, deploy, and control one million robots simultaneously.

We are in times that witness the dawn of a new science: the science of automated vehicle traffic. Although fully automated vehicles are still rare in our roads, we are in a position to find and predict the laws and equations of this new science. The foundations of the new science are built with the help of many areas in mathematics, like Theory of Partial Differential Equations, Numerical Analysis, Dynamical Systems, Fluid Mechanics, Mathematical Physics but above all Nonlinear Control Theory.

In this talk, we discuss the design, analysis, and application of learning models in large-population games. In population games, given a set of strategies, each agent in a population selects a strategy to engage in repeated strategic interactions with others. Rather than computing and adopting the best strategy selection based on a known cost function, the agents need to learn such strategy selection from instantaneous payoffs they receive at each stage of the repeated interactions.

Data-driven, algorithmic and intelligent systems are informing, mediating and automating increasingly more parts of our daily lives as well as of our public infrastructures, services and democratic processes. Opportunities abound, ubiquitous experimentation has led to many emerging forms of undesirable and sometimes harmful system outcomes. In an effort to address algorithmic harms and injustices, a plethora of technical, ethical and policy efforts has been proposed.

We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step. Using a small-gain analysis, we prove convergence to a stationary point, with a sample complexity that improves the state of the art. The key technical challenge is in connecting the actor-critic to a perturbed gradient descent, which is often obtained by allowing for infinitely many critic steps and is not possible in single-time scale settings.

Deep neural networks have drastically changed the landscape of several engineering areas such as computer vision and natural language processing. Notwithstanding the widespread success of deep networks in these, and many other areas, it is still not well understood why deep neural networks work so well. In particular, the question of which functions can be learned by deep neural networks has remained unanswered.