Logo: University of Southern California

Events Calendar


  • NLSeminar-INFORMATION-THEORETIC PROBING WITH MINIMUM DESCRIPTION LENGTH

    Thu, Dec 03, 2020 @ 10:00 AM - 11:00 PM

    Information Sciences Institute

    Conferences, Lectures, & Seminars


    Speaker: Elena Voita, Univ of Edinburgh

    Talk Title: INFORMATION-THEORETIC PROBING WITH MINIMUM DESCRIPTION LENGTH

    Abstract: How can you know whether a model has learned to encode a linguistic property? The most popular approach to measure how well pretrained representations encode a linguistic property is to use the accuracy of a probing classifier (probe). However, such probes often fail to adequately reflect differences in representations, and they can show different results depending on probe hyperparameters. As an alternative to standard probing, we propose information-theoretic probing which measures minimum description length MDL of labels given representations. In addition to probe quality, the description length evaluates the amount of effort needed to achieve this quality. We show that 1 MDL can be easily evaluated on top of standard probe training pipelines, and 2 compared to standard probes, the results of MDL probing are more informative, stable, and sensible.



    Biography: Elena Voita is a Ph.D. student at the University of Edinburgh and University of Amsterdam supervised by Ivan Titov and Rico Sennrich, and is currently a Facebook PhD Fellow. Her research focuses on document level neural machine translation, as well as on understanding what and how neural models learn. Previously, she was a research scientist at Yandex Research and worked closely with the Yandex Translate team. She also teaches NLP at the Yandex School of Data Analysis.

    Host: Jon May and Mozhdeh Gheini

    More Info: https://nlg.isi.edu/nl-seminar/

    Webcast: https://usc.zoom.us/j/96662265166

    Location: Information Science Institute (ISI) - Virtual Only

    WebCast Link: https://usc.zoom.us/j/96662265166

    Audiences: Everyone Is Invited

    Contact: Petet Zamar

    Event Link: https://nlg.isi.edu/nl-seminar/

    Add to Google CalendarDownload ICS File for OutlookDownload iCal File

Return to Calendar