Events Calendar

NL Seminar-MACHINE LEARNING THROUGH THE INFORMATION BOTTLENECK

Speaker: Artemy Kolchinsky, Santa Fe Institute

Talk Title: MACHINE LEARNING THROUGH THE INFORMATION BOTTLENECK

Series: Natural Language Seminar

Abstract: he information bottleneck IB has been proposed as a principled way to compress a random variable, while only preserving that information which is relevant for predicting another random variable. In recent times, the IB has been proposed and challenged as a theoretical framework for understanding why and how deep learning architectures achieve good performance. I will cover: 1. an introduction to the ideas behind IB, 2. methods for implementing information-theoretic compression in neural networks + some possible applications of such methods, 3. the current status of the IB theory of deep learning, 4. recently discovered caveats that arise for IB in machine learning scenarios.

Biography: Artemy Kolchinsky is a postdoctoral fellow at the Santa Fe Institute (Santa Fe, NM). His work lies at the intersection of information theory, statistical physics, and machine learning. He is interested in using tools from statistical physics to derive fundamental bounds on the ability of real-world agents whether protocells, organisms, or computers to acquire and exploit information in adaptive ways.

Host: Emily Sheng

More Info: https://nlg.isi.edu/nl-seminar

Webcast: https://bluejeans.com/298422226

Location: Information Science Institute (ISI) - CR #1016

WebCast Link: https://bluejeans.com/298422226

Audiences: Everyone Is Invited

Contact: Peter Zamar

Event Link: https://nlg.isi.edu/nl-seminar