Logo: University of Southern California

Events Calendar


  • ECE Seminar

    Fri, Feb 23, 2024 @ 03:30 PM - 04:30 PM

    Ming Hsieh Department of Electrical and Computer Engineering

    Conferences, Lectures, & Seminars


    Speaker: Jorge F. Silva, PhD, Universidad de Chile

    Talk Title: Information Theoretic Measures for Representation Learning

    Abstract: Information-theoretic measures have been widely adopted for machine learning (ML) feature design. Inspired by this, we look at the relationship between information loss in the Shannon sense and the operation loss in the minimum probability of error (MPE) sense when considering a family of lossy representations (or encoders). In this talk, we introduce a series of results that show how adequate the adoption of mutual information (MI) is for predicting the operational quality of a representation in classification. Our findings support the observation that selecting/designing representations that capture informational sufficiency (IS) is appropriate for learning. However, we also show that this selection is rather conservative if the intended goal is achieving MPE in classification. We conclude by discussing the capacity of the information bottleneck (IB) method to achieve lossless prediction and the expressive power of digital encoders in ML.

    Biography: Information-theoretic measures have been widely adopted for machine learning (ML) feature design. Inspired by this, we look at the relationship between information loss in the Shannon sense and the operation loss in the minimum probability of error (MPE) sense when considering a family of lossy representations (or encoders). In this talk, we introduce a series of results that show how adequate the adoption of mutual information (MI) is for predicting the operational quality of a representation in classification. Our findings support the observation that selecting/designing representations that capture informational sufficiency (IS) is appropriate for learning. However, we also show that this selection is rather conservative if the intended goal is achieving MPE in classification. We conclude by discussing the capacity of the information bottleneck (IB) method to achieve lossless prediction and the expressive power of digital encoders in ML.

    Host: Dr. Eduardo Pavez

    More Information: Jorge Silva Seminar 2.23.24.pdf

    Location: Hughes Aircraft Electrical Engineering Center (EEB) - 132

    Audiences: Everyone Is Invited

    Contact: Gloria Halfacre

    Add to Google CalendarDownload ICS File for OutlookDownload iCal File

Return to Calendar