Bahman Gharesifard
Seminar Information
We consider the used-in-practice setting of actor-critic where proportional step-sizes are used for both the actor and the critic, with only one critic update with a single sample from the stationary distribution per actor step. Using a small-gain analysis, we prove convergence to a stationary point, with a sample complexity that improves the state of the art. The key technical challenge is in connecting the actor-critic to a perturbed gradient descent, which is often obtained by allowing for infinitely many critic steps and is not possible in single-time scale settings. This is a joint work with Alex Olshevsky at Boston University.
Bahman Gharesifard is currently a Professor with the Electrical & Computer Engineering Department at the University of California, Los Angeles. He was an Associate Professor, from 2019 to 2021, and an Assistant Professor, from 2013 to 2019, with the Department of Mathematics and Statistics at Queen's University. He was an Alexander von Humboldt research fellow with the Institute for Systems Theory and Automatic Control at the University of Stuttgart in 2019-2020. He held postdoctoral positions with the Department of Mechanical and Aerospace Engineering at University of California, San Diego 2009-2012 and with the Coordinated Science Laboratory at the University of Illinois at Urbana-Champaign from 2012- 2013. He received the 2019 CAIMS-PIMS Early Career Award, jointly awarded by the Canadian Applied & Industrial Math Society and the Pacific Institute for the Mathematical Sciences, a Humboldt research fellowship for experienced researchers from the Alexander von Humboldt Foundation in 2019, an NSERC Discovery Accelerator Supplement in 2019, and the SIAG/CST Best SICON Paper Prize 2021. He was a finalist (as an advisor) for the Best Student Paper Award at the American Control Conference in 2017. He received the Engineering and Applied Science First Year Instructor Teaching Award in 2015 and 2017. He has served on the Conference Editorial Board of the IEEE Control Systems Society and IEEE Control System Letters and is currently an Associate Editor for the IEEE Transactions on Network Control Systems. His research interests include systems and control, machine learning, distributed control and optimization, social and economic networks, game theory, geometric control theory, geometric mechanics, and Riemannian geometry.