SUNMONTUEWEDTHUFRISAT
Events for December
-
Data-Driven Control
Wed, Dec 05, 2018 @ 12:00 PM - 01:00 PM
Ming Hsieh Department of Electrical and Computer Engineering
Conferences, Lectures, & Seminars
Speaker: Paulo Tabuada, University of California Los Angeles
Talk Title: Data-Driven Control
Series: Center for Cyber-Physical Systems and Internet of Things
Abstract: More than a decade ago Fliess and co-workers proposed model-free control as a possible answer to the inherent difficulties in controlling non-linear systems. Their key insight was that by using a sufficiently high sampling rate we can use a simple linear model for control purposes thereby trivializing nonlinear controller design. Although controllers based on linear approximations of nonlinear systems are ubiquitous in industry, providing formal guarantees for such designs has remained a challenge. In this talk we re-interpret Fliess work as data-driven control and identify a set of assumptions enabling us to mathematically prove that a model is not necessary to control non-linear systems. We illustrate the usefulness and applicability of the results via experimental results and conclude by speculating about the right mix of model-based and data-driven design in the context of Cyber-Physical Systems.
Biography: Paulo Tabuada was born in Lisbon, Portugal, one year after the Carnation Revolution. He received his "Licenciatura" degree in Aerospace Engineering from Instituto Superior Tecnico, Lisbon, Portugal in 1998 and his Ph.D. degree in Electrical and Computer Engineering in 2002 from the Institute for Systems and Robotics, a private research institute associated with Instituto Superior Tecnico. Between January 2002 and July 2003 he was a postdoctoral researcher at the University of Pennsylvania. After spending three years at the University of Notre Dame, as an Assistant Professor, he joined the Electrical Engineering Department at the University of California, Los Angeles, where he established and directs the Cyber-Physical Systems Laboratory. Paulo Tabuada's contributions to cyber-physical systems have been recognized by multiple awards including the NSF CAREER award in 2005, the Donald P. Eckman award in 2009 and the George S. Axelby award in 2011. In 2009 he co-chaired the International Conference Hybrid Systems: Computation and Control (HSCC'09) and in he was program co-chair for the 3rd IFAC Workshop on Distributed Estimation and Control in Networked Systems (NecSys'12). He currently serves as associate editor for the IEEE Transactions on Automatic Control and his latest book, on verification and control of hybrid systems, was published by Springer in 2009.
Host: Paul Bogdan
Location: Hughes Aircraft Electrical Engineering Center (EEB) - 132
Audiences: Everyone Is Invited
Contact: Talyia White
-
Zap Meets Momentum: New Stochastic Approximation Algorithms and Applications to Reinforcement Learning
Wed, Dec 12, 2018 @ 12:00 PM - 01:00 PM
Ming Hsieh Department of Electrical and Computer Engineering
Conferences, Lectures, & Seminars
Speaker: Adithya Devaraj, University of Florida
Talk Title: Zap Meets Momentum: New Stochastic Approximation Algorithms and Applications to Reinforcement Learning
Series: Center for Cyber-Physical Systems and Internet of Things
Abstract: Stochastic approximation algorithms are used to approximate solutions to fixed point equations that involve expectations of functions with respect to possibly unknown distributions. Among many algorithms in machine learning, reinforcement learning algorithms such as TD- and Q-learning are two of its most famous applications.
This talk will provide an overview of stochastic approximation, with focus on optimizing the rate of convergence. Based on this general theory, the well known slow convergence of Q-learning is explained: the variance of the algorithm is typically infinite.
Three new Q-learning algorithms are introduced to dramatically improve performance:
(i) The Zap Q-learning algorithm that has provably optimal asymptotic variance, and resembles the Newton-Raphson method in a deterministic setting
(ii) The PolSA algorithm that is based on Polyak's momentum technique, but with a specialized matrix momentum, and
(iii) The NeSA algorithm based on Nesterov's acceleration technique
Analysis of (ii) and (iii) require entirely new analytic techniques. One approach is via coupling: conditions are established under which the parameter estimates obtained using the PolSA algorithm couple with those obtained using the Newton-Raphson based algorithm. Numerical examples confirm this behavior and the remarkable performance of these algorithms.
Biography: Adithya Devaraj is a Ph.D. student at the University of Florida where he works with Prof. Sean Meyn. The focus of his research has been variance reduction in stochastic approximation algorithms with application to reinforcement learning. He has held visiting/research positions at the Indian Institute of Science, Bangalore, Inria, Paris, and the Simons Institute for the Theory of Computing at UC Berkeley.
Host: Rahul Jain
Location: Ronald Tutor Hall of Engineering (RTH) - 105
Audiences: Everyone Is Invited
Contact: Talyia White
