-
PhD Thesis Proposal - Hrayr Harutyunyan
Thu, Apr 21, 2022 @ 11:00 AM - 12:30 PM
Thomas Lord Department of Computer Science
University Calendar
Title: On information captured by neural networks: connections with memorization, generalization, and learning dynamics
Abstract:
Despite their enormous capacity, modern neural networks generalize well. In this thesis proposal we use ideas from information theory to address various aspects of this phenomenon. We show that reducing label-noise information in network weights reduces memorization and improves generalization. We propose definitions for information content of data and introduce an efficient algorithm for estimating it. These definitions allow us to quantify amount of memorization of particular examples. Finally, we derive information-theoretic generalization gap bounds that depend on average information content of a single example. We demonstrate that these bounds are non-vacuous in the practical scenarios for deep learning.
Committee:
Aram Galstyan (advisor, CS)
Greg Ver Steeg (advisor, CS)
Haipeng Luo (CS)
Bistra Dilkina (CS)
Mahdi Soltanolkotabi (EE)
Location: https://usc.zoom.us/j/95264586192?pwd=WEhERDNJcGJBblFCV2RwY29IWnJhQT09
Date: Thursday, April 21, 11:00am-12:30pm.WebCast Link: https://usc.zoom.us/j/95264586192?pwd=WEhERDNJcGJBblFCV2RwY29IWnJhQT09
Audiences: Everyone Is Invited
Contact: Lizsl De Leon
