Wed, May 18, 2022 @ 03:00 PM - 05:00 PM
PhD Candidate: Rob Brekelmans
Title: Information Geometry of Annealing Paths for Inference and Estimation
Date: Wednesday, May 18, 2022, 3:00 PM PST
Location: Seeley G Mudd (SGM) Room 226 or https://usc.zoom.us/j/91562244468
Committee: Greg Ver Steeg, Aram Galstyan, Aiichiro Nakano, Assad Oberai
Estimating normalization constants and (log) marginal likelihoods is a fundamental problem in probabilistic machine learning, playing a role in maximum likelihood learning, variational inference, and estimation of information theoretic quantities. Importance sampling, where samples from a tractable proposal distribution are reweighted based on their probability under the target density, is at the heart of many successful solutions including the evidence lower bound (ELBO), importance weighted-autoencoder (IWAE), and annealed importance sampling (AIS).
In this thesis, we provide unifying perspectives on these methods and propose methodological improvements. We propose a general approach for deriving extended state-space importance sampling bounds, leading to novel AIS and energy-based methods which can accurately estimate large values of mutual information. We consider extended state space bounds in the context of variational inference, and finally propose a new one-parameter family of annealing paths which generalize the ubiquitous geometric averaging path and can improve estimation performance on example tasks.
Location: Seeley G. Mudd Building (SGM) - 226
WebCast Link: https://usc.zoom.us/j/91562244468
Audiences: Everyone Is Invited
Contact: Lizsl De Leon