Fri, Feb 07, 2020 @ 11:00 AM - 12:00 PM
Information Sciences Institute
Conferences, Lectures, & Seminars
Speaker: Artemy Kolchinsky, Santa Fe Institute
Talk Title: MACHINE LEARNING THROUGH THE INFORMATION BOTTLENECK
Series: Natural Language Seminar
Abstract: he information bottleneck IB has been proposed as a principled way to compress a random variable, while only preserving that information which is relevant for predicting another random variable. In recent times, the IB has been proposed and challenged as a theoretical framework for understanding why and how deep learning architectures achieve good performance. I will cover: 1. an introduction to the ideas behind IB, 2. methods for implementing information-theoretic compression in neural networks + some possible applications of such methods, 3. the current status of the IB theory of deep learning, 4. recently discovered caveats that arise for IB in machine learning scenarios.
Biography: Artemy Kolchinsky is a postdoctoral fellow at the Santa Fe Institute (Santa Fe, NM). His work lies at the intersection of information theory, statistical physics, and machine learning. He is interested in using tools from statistical physics to derive fundamental bounds on the ability of real-world agents whether protocells, organisms, or computers to acquire and exploit information in adaptive ways.
Host: Emily Sheng
More Info: https://nlg.isi.edu/nl-seminar
WebCast Link: https://bluejeans.com/298422226
Audiences: Everyone Is Invited
Contact: Peter Zamar