Logo: University of Southern California

Events Calendar

  • NL Seminar-Modular and Composable Transfer Learning

    Thu, Nov 03, 2022 @ 11:00 AM - 12:00 PM

    Information Sciences Institute

    Conferences, Lectures, & Seminars

    Speaker: Jonas Pfeiffer, Google

    Talk Title: Modular and Composable Transfer Learning

    Series: NL Seminar

    Abstract: REMINDER
    Meeting hosts only admit guests that they know to the Zoom meeting. Hence, you are highly encouraged to use your USC account to sign into Zoom.

    If you are an outside visitor, please inform us at nlg DASH seminar DASH host AT isi DOT edu beforehand so we will be aware of your attendance and let you in.

    In person attendance will be permitted for USC ISI faculty, staff, students only. Open to the public virtually via the zoom link and online.

    With pre-trained transformer-based models continuously increasing in size, there is a dire need for parameter-efficient and modular transfer learning strategies. In this talk, we will touch base on adapter-based fine-tuning, where instead of fine-tuning all weights of a model, small neural network components are introduced at every layer. While the pre-trained parameters are frozen, only the newly introduced adapter weights are fine-tuned, achieving an encapsulation of the down-stream task information in designated parts of the model. We will demonstrate that adapters are modular components which can be composed for improvements on a target task and how they can be used for out of distribution generalization on the example of zero shot cross-lingual transfer. Finally, we will discuss how adding modularity during pre training can mitigate catastrophic interference and consequently lift the curse of multilinguality.

    Biography: Jonas Pfeiffer is a Research Scientist at Google Research. He is interested in modular representation learning in multi task, multilingual, and multi-modal contexts, and in low resource scenarios. He worked on his PhD at the Technical University of Darmstadt, was a visiting researcher at the New York University and a Research Scientist Intern at Meta Research. Jonas has received the IBM PhD Research Fellowship award for 2021/2022. He has given numerous invited talks at academia, industry and ML summer schools, and has co-organized multiple workshops on multilinguality and multimodality

    Host: Jon May and Meryem M'hamdi

    More Info: https://nlg.isi.edu/nl-seminar/

    Webcast: https://www.youtube.com/watch?v=hrGOb4okvI0

    Location: Information Science Institute (ISI) - Virtual and ISI-Conf Rm#689

    WebCast Link: https://www.youtube.com/watch?v=hrGOb4okvI0

    Audiences: Everyone Is Invited

    Contact: Pete Zamar

    Event Link: https://nlg.isi.edu/nl-seminar/


Return to Calendar