Center of Autonomy and AI, Center for Cyber-Physical Systems and the Internet of Things, and Ming Hsieh Institute for Electrical & Computer Engineering Joint Seminar Series: Dengwang Tang (USC)

Speaker: Dengwang Tang, University of Southern California

Talk Title: Informed Posterior Sampling Based Algorithms for Markov Decision Processes

Series: Center of Autonomy and AI, Center for Cyber-Physical Systems and the Internet of Things, and Ming Hsieh Institute for Electrical & Computer Engineering Joint Seminar Series

Abstract: The traditional paradigm of RL often features an agent who learns to control the system only through interaction. However, such a paradigm can be impractical since
learning can be very slow. In many engineering applications, there's often an offline dataset available before the application of the online learning algorithm. We proposed
the informed posterior sampling-based reinforcement learning (iPSRL) to use offline datasets to bootstrap online RL algorithms in both episodic and continuing MDP
learning problems. In this algorithm, the learning agent forms an informed prior with the offline data along with the knowledge about the offline policy that generated the data.
This informed prior is then used to initiate the posterior sampling procedure. Through a novel prior-dependent regret analysis of the posterior sampling procedure, we showed
that when the offline data is informative enough, the iPSRL algorithm can significantly reduce the learning regret compared to the baseline. Based on iPSRL, we then
proposed the more practical iRLSVI algorithm and we showed that in episodic MDP learning problems, it can significantly reduce regret via empirical results.

Biography: Dengwang Tang is currently a postdoctoral researcher at University of Southern California. He obtained his B.S.E in Computer Engineering from University of Michigan,
Ann Arbor in 2016. He earned his Ph.D. in Electrical and Computer Engineering (2021), M.S. in Mathematics (2021), and M.S. in Electrical and Computer Engineering (2018) all
from University of Michigan, Ann Arbor. Before joining USC, he was a postdoctoral researcher at University of California, Berkeley. His research interests involve control
and learning algorithms in stochastic dynamic systems, multi-agent systems, queuing theory, and dynamic games.

Host: Pierluigi Nuzzo

Location: Hughes Aircraft Electrical Engineering Center (EEB) - 132

Audiences: Everyone Is Invited

Contact: CS Events