CS Colloquium: Matus Telgarsky (University of Illinois, Urbana-Champaign) - Searching for the implicit bias of deep learning
Tue, Mar 07, 2023 @ 11:00 AM - 12:00 PM
Thomas Lord Department of Computer Science
Conferences, Lectures, & Seminars
Speaker: Matus Telgarsky, University of Illinois, Urbana-Champaign
Talk Title: Searching for the implicit bias of deep learning
Series: CS Colloquium
Abstract: What makes deep learning special --- why is it effective in so many settings where other models fail? This talk will present recent progress from three perspectives. The first result is approximation-theoretic: deep networks can easily represent phenomena that require exponentially-sized shallow networks, decision trees, and other classical models. Secondly, I will show that their statistical generalization ability --- namely, their ability to perform well on unseen testing data --- is correlated with their prediction margins, a classical notion of confidence. Finally, comprising the majority of the talk, I will discuss the interaction of the preceding two perspectives with optimization: specifically, how standard descent methods are implicitly biased towards models with good generalization. Here I will present two approaches: the strong implicit bias, which studies convergence to specific well-structured objects, and the weak implicit bias, which merely ensures certain good properties eventually hold, but has a more flexible proof technique.
This lecture satisfies requirements for CSCI 591: Research Colloquium
Biography: Matus Telgarsky is an assistant professor at the University of Illinois, Urbana-Champaign, specializing in deep learning theory. He was fortunate to receive a PhD at UCSD under Sanjoy Dasgupta. Other highlights include: co-founding, in 2017, the Midwest ML Symposium (MMLS) with Po-Ling Loh; receiving a 2018 NSF CAREER award; and organizing two Simons Institute programs, one on deep learning theory (summer 2019), and one on generalization (fall 2024).
Host: Vatsal Sharan
Location: Olin Hall of Engineering (OHE) - 132
Audiences: Everyone Is Invited
Contact: Assistant to CS chair