Events Calendar

CSC/CommNetS-MHI Seminar: Thomas Zhang

Speaker: Thomas Zhang, University of Pennsylvania

Talk Title: Guarantees for Representation Learning: Distribution Shift, Optimization, Fewer Samples

Abstract: In many areas of machine learning, it is the general understanding that broadly useful features can be extracted from data across different tasks or domains. This forms the key intuition behind the "pre-train then fine-tune" paradigm, and more generally representation learning (learning the feature mappings) and transfer learning (downstream performance on unseen tasks). Naturally, there has been significant effort to document the benefit of using diverse multi-task data both empirically and theoretically. However, prior works impose various assumptions that greatly affect their applicability, especially in settings involving data generated by dynamical systems, e.g. robotics and control.
In this talk, I will introduce the multi-task representation learning problem, and walk through the pathologies arising from sequential settings, previewed in the talk title. I will then present our recent results addressing many of these issues. In particular, we provide generalization guarantees which illustrate the benefit of learning a shared representation across domains, remaining valid even when there are too few samples to solve each task individually. We then show that optimizing for the representation is surprisingly hard, requiring critical algorithmic modifications to ensure convergence. Lastly, I will show how these results, descended from iid learning, can be lifted to dynamical systems to ensure closed-loop performance.

Biography: Thomas Zhang is a 5th-year PhD student at the University of Pennsylvania advised by Prof. Nikolai Matni. His research interests involve a combination of dynamical systems, statistical learning, and control theory. Prior to Penn, Thomas received BSc’s in Mathematics and Statistics & Data Science from Yale University, where he then spent a year as a research scientist in the Applied Mathematics Program.

Host: Dr. Stephen Tu, stephen.tu@usc.edu

More Information: 2024.09.16 CSC Seminar - Thomas Zhang.pdf

Location: Hughes Aircraft Electrical Engineering Center (EEB) - EEB 132

Audiences: Everyone Is Invited

Contact: Miki Arlen